United  States  Government  Accountability  Office 


GAO 


Report  to  Congressional  Committees 


DEFENSE 

ACQUISITION 

WORKFORCE 

DOD’s  Training 
Program 

Demonstrates  Many 
Attributes  of 
Effectiveness,  but 
Improvement  Is 
Needed 


GAO 

Accountability  *  Integrity  *  Reliability 


GAO-11-22 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

Q^irp  2010  ^  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2010  to  00-00-2010 

4.  TITLE  AND  SUBTITLE 

Defense  Acquisition  Workforce:  DOD’s  Training  Program  Demonstrates 
Many  Attributes  of  Effectiveness,  but  Improvement  Is  Needed 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

U.S.  Government  Accountability  Office, 441  G  Street 

NW, Washington, DC, 20548 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  OS 

unclassified  unclassified  unclassified  Report  (SAR) 

47 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


October  2010 


L  GAO  DEFENSE  ACQUISITION  WORKFORCE 

DOD's  Training  Program  Demonstrates  Many 
Attributes  of  Effectiveness,  but  Improvement  Is 
Needed 

Highlights  of  GAO-1 1  -22,  a  report  to 
congressional  committees 


Accountability  •  Integrity  •  Reliability 

Highlights 


Why  GAO  Did  This  Study 


What  GAO  Found 


The  President  has  announced  his 
intention  to  improve  the  acquisition 
process,  particularly  given  the  half  a 
trillion  dollars  the  federal 
government  spent  in  fiscal  year  2009 
on  acquiring  goods  and  services.  The 
Department  of  Defense  (DOD)  spent 
$384  billion  in  fiscal  year  2009  on 
goods  and  services — double  what  it 
spent  in  2001.  A  high-quality 
workforce  with  the  right 
competencies  and  skill  sets  will  be 
critical  to  improving  DOD 
acquisitions. 

GAO  was  mandated  to  determine  the 
efficacy  of  DOD’s  certification 
training  for  its  acquisition  workforce. 
GAO  assessed  (1)  DOD’s  capability  to 
provide  certification  training,  (2)  the 
extent  that  such  training  reaches 
members  of  the  workforce,  and 
(3)  the  extent  that  previous  training 
recommendations  have  been 
implemented.  To  conduct  this  work, 
GAO  compared  DOD’s  certification 
training  to  GAO  guidance  for 
effective  training  programs  and 
analyzed  policies,  data,  and  previous 
reports  on  acquisition  training. 


DOD’s  certification  training  program — provided  by  the  Defense  Acquisition 
University  (DAU) — generally  demonstrates  the  capability  to  provide  effective 
training,  though  some  attributes  of  an  effective  training  program  are  lacking. 
DAU  ensures  that  strategic  and  tactical  changes  are  promptly  incorporated 
into  training;  uses  centralized  and  decentralized  training  approaches  in  design 
and  development;  collects  data  during  implementation  to  ensure  feedback  on 
its  training  programs;  and  analyzes  its  training  during  evaluation.  However, 
DOD  lacks  complete  information  on  the  skill  sets  of  the  current  acquisition 
workforce  and  does  not  have  outcome-based  metrics  to  assess  results 
achieved  in  enhancing  workforce  proficiency  and  capability  through  training 
efforts.  In  2009,  GAO  recommended  that  the  Secretary  of  Defense  identify  and 
update  on  an  ongoing  basis  the  number  and  skill  sets  of  the  total  acquisition 
workforce — including  civilian,  military,  and  contractor  personnel — that  the 
department  needs  to  fulfill  its  mission.  DOD  agreed  and  to  date  has  completed 
about  one-fifth  of  its  workforce  competency  assessments. 


At  the  end  of  fiscal  year  2009,  90  percent  of  DOD’s  acquisition  workforce 
personnel  had  completed  required  certification  training  or  were  within 
required  time  frames  to  do  so,  according  to  DAU  data.  However,  DAU  reports 
that  it  cannot  provide  for  all  training  requested  for  the  entire  acquisition 
workforce.  DAU  has  offered  more  courses  in  recent  years,  and  high-priority 
personnel — those  needing  to  complete  classes  for  certification  in  their  current 
position — constitute  the  majority  in  DAU  classes.  DAU  plans  the  number  and 
location  of  its  classes  based  on  data  that  DOD  officials  noted  are  generally 
incomplete  when  submitted,  and  DAU  must  adapt  during  the  year  to  support 
new  requirements  as  they  are  identified.  DAU  has  identified  the  need  for  a 
new,  integrated  student  information  system  that  will  provide  better  insight 
into  the  workforce  it  supports  and  is  in  the  early  stages  of  its  procurement. 


What  GAO  Recommends 

GAO  recommends  DOD  establish 
milestones  for  developing  metrics  to 
measure  how  certification  training 
improves  acquisition  workforce 
capability  and  a  time  frame  for 
acquiring  and  implementing  an 
integrated  information  system.  DOD 
concurred  with  the  second  but  not 
the  first  recommendation.  GAO 
continues  to  believe  DOD  needs  to 
develop  additional  metrics. 

View  GAO-1 1  -22  or  key  components. 

For  more  information,  contact  John  Needham 
at  (202)  512-4841  or  NeedhamJK1@gao.gov. 


DOD  reports  that  most  of  the  training-related  recommendations  from  previous 
reviews — the  Gansler  Commission,  the  Panel  on  Contracting  Integrity,  and  a 
prior  GAO  report — have  been  fully  implemented  and  some  actions  are  still 
under  way.  DOD  has  either  fully  or  partially  implemented  15  of  the  19 
recommendations  GAO  reviewed.  Both  the  Army  and  the  Office  of  the 
Secretary  of  Defense  have  taken  steps  to  respond  to  the  Gansler  Commission 
recommendations.  Most  of  the  recommendations  made  by  the  Panel  on 
Contracting  Integrity  have  been  implemented,  with  the  exception  of  two 
recommendations  related  to  assessing  guidance  and  reviewing  a  specific 
training  topic.  GAO  made  four  recommendations  pertaining  to  the  Defense 
Contract  Audit  Agency’s  government  auditing  standards  training  and 
expertise,  of  which  one  has  been  partially  implemented  and  three  have  not 
been  implemented,  but  some  actions  have  been  taken. 
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GAO 

^^Accountability  *  Integrity  *  Reliability 

United  States  Government  Accountability  Office 
Washington,  DC  20548 


October  28,  2010 
Congressional  Committees 

The  President  has  announced  the  administration’s  intention  to  achieve 
improvements  in  the  acquisition  system.  Current  fiscal  challenges 
highlight  the  need  to  scrutinize  the  processes  used  by  the  federal 
government  to  spend  half  a  trillion  dollars  in  fiscal  year  2009  acquiring 
goods  and  services.  The  Department  of  Defense’s  (DOD)  spending  on 
goods  and  services  has  more  than  doubled  since  2001  to  nearly 
$384  billion;1  however,  the  number  of  acquisition  personnel  has  remained 
relatively  stable.  According  to  the  Office  of  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology  and  Logistics  (USD(AT&L)),  to  successfully 
accomplish  the  acquisition  mission  and  ensure  the  best  value  for  the 
expenditure  of  public  resources,  DOD  will  place  greater  emphasis  on 
having  a  high-quality  workforce  with  the  right  competencies  and  skill  sets.2 
In  addition  to  building  the  capability  of  the  existing  workforce,  the 
Secretary  of  Defense  plans  to  augment  the  capacity  of  the  defense 
acquisition  workforce  by  increasing  its  numbers  by  20,000  employees  over 
the  next  5  years.3 


1  According  to  the  Federal  Procurement  Data  System-Next  Generation,  DOD’s  total 
obligations  in  fiscal  year  2009  were  about  $370  billion.  However,  this  figure  reflects  an 
approximately  $13.9  billion  downward  adjustment  made  by  DOD  to  correct  an 
administrative  error  made  in  fiscal  year  2008.  As  this  adjustment  significantly  affected 
DOD’s  reported  obligations  in  fiscal  year  2009,  the  $384  billion  figure  we  report  reflects 
what  DOD  would  have  reported  had  the  error  not  occurred. 

2  According  to  10  U.S.C.§  1702,  USD(AT&L)  shall  carry  out  all  powers,  functions,  and  duties 
of  the  Secretary  of  Defense  with  respect  to  the  DOD  AT&L  workforce.  DOD  Instruction 
5000.57 — Defense  Acquisition  University — lists  establishing  a  program  of  education  and 
training  standards,  requirements,  and  performance  learning  assets  for  the  civilian  and 
military  AT&L  workforce  as  one  of  the  specific  duties  of  the  USD(AT&L). 

3  DOD  defines  its  acquisition  workforce  to  include  15  career  fields,  based  on  the  Defense 
Acquisition  Workforce  Improvement  Act  (Pub.  L.  No.  101-510,  §  1202(a)  (1990)).  These 
career  fields  are  program  management;  contracting;  industrial/contract  property 
management;  purchasing;  facilities  engineering;  production,  quality,  and  manufacturing; 
business — cost  estimating;  business — financial  management;  lifecycle  logistics; 
information  technology;  systems  planning,  research,  development,  and  engineering — 
systems  engineering;  systems  planning,  research,  development  and  engineering — program 
systems  engineer;  systems  planning,  research,  development  and  engineering — science  and 
technology  manager;  test  and  evaluation;  and  auditing. 
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In  1990  the  Defense  Acquisition  Workforce  Improvement  Act  (DAWLA) 
established  career  paths  for  the  improvement  of  training  for  the 
acquisition  workforce.4  Given  the  enduring  critical  role  of  the  acquisition 
workforce  in  achieving  the  administration’s  goal  of  improving  federal 
acquisition,  it  is  important  they  have  the  skills  and  support  needed  to  do 
their  jobs.  Accordingly,  in  the  National  Defense  Authorization  Act  for 
Fiscal  Year  20 10, 5  Congress  mandated  that  GAO  study  the  efficacy  of 
DOD’s  training  for  its  acquisition  and  audit  workforce.  As  agreed  with 
your  respective  offices,  we  assessed  (1)  DOD’s  capability  to  provide 
defense  acquisition  workforce  certification  training,  (2)  the  extent  that 
such  training  reaches  members  of  DOD’s  acquisition  workforce,  and 
(3)  the  extent  that  training  recommendations  from  previous  reviews, 
including  the  Gansler  Commission,  have  been  implemented.6 

To  determine  DOD’s  capability  to  provide  acquisition  certification  training, 
we  focused  on  training  for  DOD  personnel  covered  under  DAWIA.7  We 


4  For  the  purposes  of  this  report,  the  personnel  covered  under  the  Defense  Acquisition 
Workforce  Improvement  Act  (DAWIA)  will  be  referred  to  as  the  acquisition  workforce. 
Under  DAWIA,  career  fields,  such  as  program  management,  and  certification  of  personnel 
in  those  career  fields,  were  established  to  provide  the  necessary  skills.  To  be  certified, 
acquisition  workforce  members  must  meet  mandatory  standards  for  the  career  field  level 
(I,  II,  or  III)  required  for  their  position.  For  DAWIA  certification  in  each  career  field,  there 
are  requirements  in  three  areas — education,  experience,  and  training.  We  compared  DOD’s 
certification  training  program  for  14  of  the  15  career  fields  with  the  attributes  of  effective 
training  and  development  programs  identified  in  GAO’s  guide  (GAO,  Human  Capital:  A 
Guide  for  Assessing  Strategic  Training  and  Development  Efforts  in  the  Federal 
Government,  GAO-04-546G  (Washington,  D.C.:  Mar.  2004))  and  listed  in  appendix  II.  We 
were  not  able  to  apply  these  criteria  to  the  auditing  career  field  because  the  Defense 
Contract  Audit  Agency  (DCAA)  lacks  a  strategic  plan  (GAO,  DCAA  Audits:  Widespread 
Problems  with  Audit  Quality  Require  Significant  Reform,  GAO-09-468  (Washington,  D.C.: 
Sept.  23,  2009)).  A  strategic  plan  is  a  key  document  for  assessing  training  programs  using 
the  strategic  training  efforts  attributes. 

5  Pub.  L.  No.  111-84,  §  1108  (b)  (2)  (2009). 

6  The  Commission  on  Army  Acquisition  and  Program  Management  in  Expeditionary 
Operations — commonly  referred  to  as  the  Gansler  Commission — assessed  aspects  of 
recent  expeditionary  operations,  including  training,  to  ensure  that  the  Army  is  properly 
equipped  for  future  operations.  The  commission’s  October  2007  report  made  four 
overarching  recommendations,  including  providing  training  and  tools  for  overall 
contracting  activities  in  expeditionary  operations. 

'  This  report  addresses  the  training  component  of  acquisition  workforce  certification,  not 
the  education  and  experience  components.  Further,  we  limited  our  review  of  the  auditing 
career  field  to  DCAA  auditors  because  they  are  the  only  DOD  auditors  covered  by  DAWIA. 
Additionally,  we  have  ongoing  work  to  review  training  for  DOD  personnel  with  acquisition 
responsibilities  in  contingency  and  non-contingency  environments  but  who  are  not  covered 
by  DAWIA. 
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compared  DOD’s  certification  training  programs  and  processes  with  the 
attributes  of  effective  training  and  development  programs  identified  in  our 
previous  work  on  strategic  training  and  development  efforts  in  the  federal 
government.8  We  interviewed  officials  at  the  Defense  Acquisition 
University  (DAU)  and  Defense  Contract  Audit  Institute  (DCAI)  to  obtain 
an  understanding  of  their  training  programs  and  processes,  and  we 
obtained  documents — such  as  briefings,  guidance,  strategic  plans,  and 
course  catalogs — describing  the  training  programs  and  processes.  In 
addition,  we  selected  a  nongeneralizable  sample  of  one  command  from 
each  military  service  near  a  DAU  regional  office — based  on  level  of 
procurement  dollars  spent  and  number  of  DAU  courses  completed  in 
fiscal  years  2008  through  2009 — and  program  offices  within  those 
commands  that  would  likely  have  a  large  cross-section  of  acquisition 
workforce  personnel  to  obtain  customer  perspectives  on  the  effectiveness 
and  usefulness  of  the  DAU  training  and  to  determine  the  use  of 
supplementary  training.  We  also  interviewed  Defense  Contract 
Management  Agency  officials  to  obtain  their  views  on  DAU  training. 
Finally,  we  visited  a  nongeneralizable  sample  of  two  Defense  Contract 
Audit  Agency  (DCAA)  locations  to  obtain  customer  perspectives  on  DCAI 
training.  We  did  not  examine  the  appropriateness  of  the  certification 
training  itself  nor  the  content  of  courses  required  for  certification. 

To  assess  the  extent  to  which  acquisition  training  reaches  acquisition 
personnel,  we  reviewed  DAU  and  DCAI  policies  on  determining  training 
requirements,  allocating  resources,  and  scheduling  classes.  We  collected 
and  analyzed  defense  acquisition  workforce  and  training  data  maintained 
in  the  AT&L  Data  Mart  system9  used  by  DAU  to  determine  course  demand 
and  certification  status  of  acquisition  workforce  members.  This  provided 
an  understanding  of  the  number  of  class  requests  received,  class  seats 
scheduled,  and  students  who  registered  and  completed  these  courses  in 
past  fiscal  years.  We  assessed  the  reliability  of  these  data  by  reviewing 
data  query  information  and  interviewing  knowledgeable  officials  who 
collect  and  use  these  data.  We  determined  that  the  data  were  sufficiently 
reliable  for  the  purposes  of  this  report.  In  addition,  we  interviewed  DAU 


8  GAO,  Human  Capital:  A  Guide  for  Assessing  Strategic  Training  and  Development 
Efforts  in  the  Federal  Government,  GAO-04-546G  (Washington,  D.C.:  Mar.  2004). 

9  The  Acquisition  Technology  and  Logistics  Workforce  Data  Mart  was  designed  to  be  the 
single  source  of  acquisition  data  for  career  and  workforce  management  for  both  civilian 
and  military  personnel.  Information  contained  within  AT&L  Data  Mart  comes  from 
multiple,  external  military  and  civilian  data  sources. 
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officials  and  obtained  budget  documents  to  determine  DOD’s  use  of  the 
Defense  Acquisition  Workforce  Development  Fund  for  training. 

To  determine  the  extent  to  which  training  recommendations  from 
previous  reviews  have  been  implemented,  we  identified  previous  reviews 
and  interviewed  and  obtained  documentation  from  agency  officials  on  the 
status  of  DOD’s  implementation  of  the  training  recommendations. 

We  conducted  this  performance  audit  from  December  2009  to  September 
2010  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives.  See  appendix  I  for 
additional  details  on  our  objectives,  scope,  and  methodology. 


Background 


Congress  passed  DAWIA  in  1990  to  ensure  effective  and  uniform 
education,  training,  and  career  development  of  members  of  the  acquisition 
workforce.10  Accordingly,  the  act  established  DAU  to  provide  training  for 
the  DOD  acquisition  workforce  and  charged  DOD  officials  with 
designating  acquisition  positions,  setting  qualification  requirements,  and 
establishing  policies  and  procedures  for  training  the  acquisition 
workforce.  DOD,  as  part  of  implementing  DAWIA,  established  career 
fields,  such  as  program  management  (See  table  1). 


10  Pub.  L.  No.  101-510,  §  1202(a)  (1990). 


Page  4 


GAO-11-22  Defense  Acquisition  Workforce 


Table  1 :  Defense  Acquisition  Workforce  Personnel  by  Career  Field  in  Fiscal  Year 
2009 


Career  field 

Army 

Navy/Marine 

Corps 

Air 

Force 

Defense 

agencies 

Total 

Auditing3 

0 

0 

0 

3,777 

3,777 

Business— cost 
estimating  and  financial 
management4 

2,771 

2,286 

1,845 

360 

7,262 

Contracting 

8.391 

5,516 

7,443 

6,305 

27,655 

Facilities  engineering 

719 

4,683 

6 

12 

5,420 

Industrial/contract 
property  management 

92 

73 

29 

281 

475 

Information  technology 

1,843 

1,240 

966 

309 

4,358 

Life-cycle  logistics 

7,952 

4,784 

1,989 

127 

14,852 

Production,  quality,  and 
manufacturing 

1,930 

2,064 

389 

4,640 

9,023 

Program  management 

3,452 

4,598 

4,461 

911 

13,422 

Purchasing 

330 

567 

146 

195 

1,238 

System  planning, 
research,  development, 
and  engineering 
(SPRDE)— science  and 
technology  manager" 

204 

243 

51 

125 

623 

SPRDE  — systems 
engineering 

10,175 

18,003 

7,113 

1,179 

36,470 

SPRDE  — program 
system  engineer 

33 

82 

84 

35 

234 

Test  and  Evaluation 

2,235 

2,833 

2,630 

194 

7,892 

Unknown 

229 

0 

22 

151 

402d 

Total 

40,356 

46,972 

27,174 

18,601 

133,103 

Source:  AT&L  Workforce  Data  Mart. 


aDAU  oversees  the  delivery  of  certification  training  for  all  of  the  DAWIA  career  fields  except  auditing. 
The  Director,  Defense  Contract  Audit  Agency,  underthe  Undersecretary  of  Defense  (Comptroller), 
provides  a  program  of  education,  training,  career  development,  and  workforce  management  at  the 
Defense  Contract  Audit  Institute  forthe  auditor  career  field,  in  coordination  with  DAU,  as  appropriate. 

bE ffective  October  1,  2009,  the  Business— cost  estimating  and  financial  management  career  field  was 
split  into  two  career  fields:  Business— cost  estimating  and  Business— financial  management,  bringing 
the  number  of  career  fields  to  15. 

"System  planning,  research,  development,  and  engineering  (SPRDE)  has  three  distinct  career  paths: 
(1)  SPRDE  — science  and  technology  manager,  (2)  SPRDE  — systems  engineering,  and  (3)  SPRDE— 
program  system  engineer. 

d402  records  submitted  with  no  career  field  data. 
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The  act  also  required  DOD  to  establish  career  paths,  referred  to  by  DOD  as 
certification  requirements,  for  the  acquisition  workforce.11  DOD  military 
services  and  defense  agencies  must  track  that  acquisition  workforce 
members  meet  mandatory  standards  established  for  level  I  (basic  or 
entry),  level  II  (intermediate  or  journeyman),  or  level  III  (advanced  or 
senior)  in  a  career  field,  such  as  contracting,  life-cycle  logistics,  and 
program  management.  DAU  is  responsible  for  certification  training  and  for 
designing,  maintaining,  and  overseeing  the  delivery  of  certification  training 
courses  at  each  level,  among  other  things.  For  each  career  field  and  level, 
there  are  requirements  in  three  areas — education,  experience,  and 
training.  Certification  requirements  are  the  same  for  civilian  and  military 
acquisition  workforce  members.  Table  2  shows  the  nature  of  certification 
training  for  one  of  the  DAWIA  career  fields — system  planning,  research, 
development,  and  engineering  (SPRDE) — systems  engineering,  as  well  as 
shows  the  education  and  experience  requirements  for  each  level  in  the 
career  field. 


11  It  is  the  responsibility  of  the  DOD  functional  leaders  to  establish,  oversee,  and  maintain 
the  education,  training,  and  experience  requirements  including  competencies  and 
certification  standards,  position  category  description(s),  and  content  of  the  DAU  courses 
as  current,  technically  accurate,  and  consistent  with  DOD  acquisition  policies.  Para.  5.5.3, 
DOD  Instruction  5000.66,  paragraph  5.5.3.  (Dec.  2005). 
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Table  2:  SPRDE — Systems  Engineering  Certification  Requirements  by  Career  Level 


Career 

level 

Training  (class  titles  and  codes) 

Education 

Experience 

1 

Fundamentals  of  Systems  Acquisition 
Management  (ACQ  101) 

Fundamentals  of  Systems  Planning, 
Research,  Development,  and 

Engineering  (SYS  101) 

Baccalaureate  or 
graduate  degree  in  a 
technical  or  scientific  field 
such  as  engineering, 
physics,  chemistry, 
biology,  mathematics, 
operations  research, 
engineering 
management,  or 
computer  science 

One  year  of  technical  experience  in  an  acquisition 
position  from  among  the  following  career  fields: 
SPRDE  — systems  engineering  (SE);  SPRDE— 
science  and  technology  manager  (S&TM); 
information  technology  (IT);  test  and  evaluation 
(T&E);  production,  quality,  and  manufacturing 
(PQM);  facilities  engineering  (FE);  program 
management  (PM);  or  life-cycle  logistics  (LCL). 
Similar  experience  gained  from  other  government 
positions  or  industry  is  acceptable  as  long  as  it 
meets  the  above  standards. 

II 

Intermediate  Systems  Acquisition,  Part  A 
(ACQ  201A) 

Intermediate  Systems  Acquisition,  Part  B 
(ACQ  201B) 

Intermediate  Systems  Planning, 

Research,  Development,  and 

Engineering,  Parti  (SYS  202) 
Intermediate  Systems  Planning, 

Research,  Development,  and 

Engineering,  Partll  (SYS  203) 

Technical  Reviews  (CLE  003) 

Same  as  level  1 

Through  September  30,  2010: 

Two  years  of  technical  experience  in  an  acquisition 
position,  including  government  or  industry 
equivalent,  from  among  the  career  fields  listed  for 
level  1 

Effective  October  1,  2010: 

Two  years  of  technical  experience  in  an  acquisition 
position.  Of  that,  at  least  1  year  in  a  SPRDE-SE, 
SPRDE-program  system  engineer  (PSE),  or 
SPRDE-S&TM  position;  remainder  may  come  from 
IT, T&E,  PQM,  PM,  orLCL 

Similar  experience  gained  from  other  government 
positions  or  industry  is  acceptable  as  long  as  it 
meets  the  above  standards. 

III 

Technical  Leadership  in  Systems 
Engineering  (SYS  302) 

Designing  for  Supportability  in  DOD 
Systems  (CLL  008) 

Same  as  level  1 

Through  September  30,  2010: 

Four  years  of  technical  experience  in  an 
acquisition  position,  to  include  government  or 
industry  equivalent,  from  among  the  career  fields 

listed  for  level  I 
Effective  October  1,  2010: 

Four  years  of  technical  experience  in  an 
acquisition  position.  Of  that,  at  least  3  years  in  a 
SPRDE-SE,  SPRDE-PSE,  or  SPRDE -S&TM 
position;  remainder  may  come  from  IT,  T&E,  PQM, 
PM,  or  LCL 

Similar  experience  gained  from  other  government 
positions  or  industry  is  acceptable  as  long  as  it 
meets  the  above  standards. 


Source:  GAO  (presentation);  DAU  Web  site  (data). 
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Besides  the  certification  training  it  offers,  DAU  approves  alternative 
certification  training  providers  based  on  a  review  by  an  independent 
organization — the  American  Council  on  Education — of  the  capability  of  a 
potential  provider  to  offer  acquisition  training  and  whether  the  provider’s 
course  content  addresses  the  DAU  course’s  learning  outcomes.12  An 
equivalent  course  provider  must  certify  annually  that  its  course  is  current 
with  the  DAU  plan  of  instruction  for  the  course.  Similarly,  DCAI  provides 
both  required  certification  training  and  supplemental  training  for  the 
auditor  career  field. 

In  addition  to  certification  training,  DAU  offers  supplemental  training  for 
each  career  field  and  for  particular  types  of  assignments.  For  example,  for 
level  II  contracting  in  contingency  or  combat  operations,  DAU  provides 
courses  such  as  a  contingency  contracting  simulation,  a  contingency 
contracting  officer  refresher,  and  a  joint  contingency  contracting  course. 
DAU  also  provides  continuous  learning  modules  online  to  provide 
acquisition  workforce  members  with  a  quick  reference  for  material 
already  introduced  and  courses  to  help  them  maintain  currency  in  their 
career  field  by  achieving  the  required  80  continuous  learning  points 
biennially.  Additionally,  DAU  provides  consulting  support  to  program 
offices,  rapid-deployment  training  on  new  initiatives,  and  training  targeted 
to  the  needs  of  acquisition  field  organizations.  DAU  also  engages  in 
knowledge-sharing  initiatives,  including  hosting  a  number  of  acquisition 
communities  of  practice  and  providing  Web-based  acquisition  policy  and 
reference  materials. 

In  March  2004,  we  issued  a  guide  for  assessing  federal  training  programs 
that  breaks  the  training  and  development  process  into  four  broad, 
interrelated  components — (1)  planning  and  front-end  analysis,  (2)  design 


12  DOD  Instruction  5000.57  requires  DAU  to  establish  a  course  equivalency  program  that 
identifies  alternatives  for  the  workforce  to  attain  required  training.  With  over  30  years  of 
experience  in  reviewing  learning  programs  offered  by  business,  government,  and  military 
organizations  for  equivalent  college  credit,  the  American  Council  on  Education  assesses  all 
potential  equivalency  providers  and  their  courses,  and,  based  on  its  recommendations, 
DAU  approves  providers — including  universities,  DOD  schools,  and  commercial  vendors — 
to  offer  courses  approved  as  equivalent  to  students  attempting  to  meet  DAU  course 
requirements. 
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and  development,  (3)  implementation,  and  (4)  evaluation.13  The  guide 
discusses  attributes  of  effective  training  and  development  programs  that 
should  be  present  in  each  of  the  components  and  identifies  practices  that 
indicate  the  presence  of  the  attribute.14  For  example,  under  the  design  and 
development  component,  to  determine  whether  an  organization  possesses 
the  attribute  of  incorporating  measures  of  effectiveness  into  courses  it 
designs,  the  guide  suggests  looking  for  practices,  such  as  (1)  clear  linkages 
between  specific  learning  objectives  and  organizational  results  and 
(2)  well-written  learning  objectives  that  are  unambiguous,  achievable,  and 
measurable.  For  a  complete  list  of  the  attributes  of  effective  training  and 
development  programs,  see  appendix  II. 

Figure  1  depicts  the  training  and  development  process  along  with  the 
general  relationships  between  the  four  components  that  help  to  produce  a 
strategic  approach  to  federal  agencies’  training  and  development 
programs.  These  components  are  not  mutually  exclusive  and  encompass 
subcomponents  that  may  blend  with  one  another.  Evaluation,  for  example, 
should  occur  throughout  the  process. 


13  GAO  developed  the  guide  through  consultations  with  government  officials  and  experts  in 
the  private  sector,  academia,  and  nonprofit  organizations;  examinations  of  laws  and 
regulations  related  to  training  and  development  in  the  federal  government;  and  reviewing 
the  sizeable  body  of  literature  on  training  and  development  issues,  including  previous  GAO 
products  on  a  range  of  human  capital  topics.  GAO,  Human  Capital:  A  Guide  for  Assessing 
Strategic  Training  and  Development  Efforts  in  the  Federal  Government,  GAO-04-546G 
(Washington,  D.C.:  Mar.  2004). 

14  According  to  the  guide,  these  practices  should  serve  as  a  guide  for  assessment  and  do  not 
comprise  a  complete  or  mandatory  “set”  of  elements  needed  in  response  to  each  question; 
their  relevance  will  vary  depending  on  each  agency’s  specific  circumstances. 
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Figure  1 :  Components  of  the  Training  and  Development  Process 


Planning/ 

Front-end  Analysis 


Develop  a  strategic 
approach  that  establishes 
priorities  and  leverages 
investments  in  training  and 
development  to  achieve 
agency  results. 


* 


Design/ 

Development  Evaluation 


•  Identify  specific  training 
and  development  initiatives 
that,  in  conjunction  with 
other  strategies,  improve 
individual  and  agency 
performance. 


Demonstrate  how  training 
and  development  efforts 
contribute  to  improved 
performance  and  results. 


* 


Implementation 


Ensure  effective  and 
efficient  delivery  of 
training  and  development 
opportunities  in  an 
environment  that  supports 
learning  and  change. 


Source:  GAO. 
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DOD’s  Certification 
Training  Program  Has 
the  Capability  to 
Provide  Training, 
Although  It  Lacks 
Some  Attributes  of 
Effective  Training 


DOD’s  acquisition  workforce  certification  training — centrally 
administered  by  DAU — has  many  attributes  of  effective  training  programs 
that  demonstrate  the  capability  to  deliver  training.  DAU’s  certification 
training  program  has  a  formal  process  in  planning  and  front-end  analysis 
to  ensure  that  strategic  and  tactical  changes  are  promptly  incorporated 
into  training;  use  of  centralized  and  decentralized  training  approaches  in 
design  and  development;  data  collection  during  implementation  to  ensure 
feedback  on  its  training  programs;  and  appropriate  analytical  approaches 
to  assess  its  training  during  evaluation.  However,  DOD  lacks  complete 
information  on  the  skill  sets  of  the  current  acquisition  workforce  for 
planning  and  front-end  analysis  and  does  not  have  metrics  to  assess 
results  achieved  in  enhancing  workforce  proficiency  and  capability 
through  training  efforts  during  evaluation.  Complete  data  on  acquisition 
skill  sets  are  needed  to  accurately  identify  workforce  gaps,  and 
appropriate  metrics  are  necessary  to  increase  the  likelihood  that  desired 
changes  will  occur  in  the  acquisition  workforce’s  skills,  knowledge, 
abilities,  attitudes,  or  behaviors. 


DOD’s  Certification 
Training  Program  Has 
Many  Attributes  of 
Effective  Training 
Programs 


DOD’s  certification  training  program  possesses  attributes  of  effective 
training  programs  in  each  of  the  four  components  of  the  training  and 
development  process.  Following  are  examples  of  the  attributes  we 
observed  in  DOD  training  categorized  by  the  components  of  effective 
training  programs. 

Planning  and  front-end  analysis:  Planning  and  front-end  analysis  can  help 
ensure  that  training  efforts  are  not  initiated  in  an  ad  hoc,  uncoordinated 
manner,  but  rather  are  strategically  focused  on  improving  performance 
toward  the  agency’s  goals.  DAU  had  processes  to  ensure  that  training 
efforts  were  coordinated  and  focused  on  improving  agency  goals.  Through 
a  formal  process  that  ensures  that  strategic  and  tactical  changes  are 
promptly  incorporated  into  training,  DAU  and  other  DOD  stakeholders 
plan  for  and  evaluate  the  effectiveness  of  DAU’s  training  efforts.  Each 
career  field  has  a  functional  leader,  a  senior  subject-matter  expert  in  the 
career  field  who  is  responsible  for  annually  certifying  that  course  content 
for  certification  is  current,  technically  accurate,  and  consistent  with  DOD 
acquisition  policies.15  Functional  leaders  are  supported  by  a  functional 


15  In  some  cases,  functional  leaders  are  responsible  for  more  than  one  career  field.  For 
example,  the  same  functional  leader  is  responsible  for  contracting,  purchasing,  and 
industrial/contract  property  management. 
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integrated  process  team  for  each  career  field,  which  consists  of  subject- 
matter  experts,  acquisition  career  management  representatives  from  the 
military  services  and  other  DOD  agencies,  and  DAU  representatives.  The 
functional  integrated  process  team  analyzes  and  reviews  data,  including 
end-of-course  evaluations,  number  of  students  completing  a  class,  wait 
lists,  and  certification  rates,  as  well  as  DOD  policy  changes  and 
recommendations  from  reviews,  such  as  the  Gansler  Commission  to 
support  functional  leaders.  DAU  designs  courses  in  accordance  with  the 
functional  leader  and  functional  integrated  process  team  decisions.  Using 
this  process,  strategic  and  tactical  changes  were  promptly  incorporated 
into  training.  For  example,  DAU  developed  and  fielded  a  new  contracting 
course  on  federal  acquisition  regulation  fundamentals  within  a  year  of 
direction  by  the  functional  leader’s  organization  to  create  it. 

Design  and  development:  In  design  and  development,  it  is  important  that 
agencies  consciously  consider  the  advantages  and  disadvantages  of  using 
centralized  and  decentralized  approaches.  Centralizing  design  can 
enhance  consistency  of  training  content  and  offer  potential  cost  savings. 
DAU  evaluates  and  uses  centralized  and  decentralized  approaches  for 
training  after  considering  the  advantages  and  disadvantages.  DAU’s 
curriculum  development  and  technologies  organizations  located  at  Fort 
Belvoir,  Virginia,  provide  centralized,  integrated  design  and  development 
of  certification  courses.  These  courses  are  then  delivered  to  the 
acquisition  workforce  by  five  regionally-oriented  campuses  and  the 
Defense  Systems  Management  College  School  of  Program  Managers.  DAU 
also  compares  training  delivery  mechanisms  to  determine  the  appropriate 
use  of  different  delivery  mechanisms  (such  as  classroom  or  computer- 
based  training)  and  to  ensure  efficient  and  cost-effective  delivery.  In 
addition,  supplementary  training  is  offered  at  the  Army,  Navy,  and  Air 
Force  commands  and  program  offices  we  visited,  as  well  as  at  the  Defense 
Contract  Management  Agency.  While  DAU  provides  a  foundation  for 
acquisition  and  career  field  knowledge  in  its  certification  training,  various 
decentralized  sources  provide  supplementary  training  more  targeted  to 
specific  jobs,  such  as  training  on  service-specific  processes  or  databases 
and  technical  topics.  Acquisition  workforce  members  at  the  commands  we 
visited  provided  the  following  examples  of  supplementary  training. 

•  The  contracting  offices  at  the  Army  Aviation  and  Missile  Command 
(AMCOM),  Alabama,  and  the  Air  Force  Aeronautical  Systems  Center 
(ASC),  Ohio,  provided  unique  training  in  the  contracting  area.  AMCOM’s 
Contracting  Center  University  teaches  employees  how  to  do  day-to-day 
tasks  associated  with  their  job,  such  as  price  analysis,  price  negotiation, 
and  how  to  use  the  Army  Materiel  Command-unique  system  for  preparing 
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contract  documents.  ASC’s  “jump  start”  program  teaches,  reinforces,  and 
supplements  DAU  certification  training  in  the  contracting  career  field  with 
illustrative  examples  not  provided  in  the  computer-based  contracting 
courses  as  well  as  offers  an  opportunity  to  interact  with  instructors  and 
other  students. 

•  The  Naval  Air  Systems  Command  (NAVAIR),  Maryland,  provides 
supplementary  training  for  DAWLA  career  fields.  For  example,  in  the 
program  management  career  field,  NAVAIR  offers  courses  in  configuration 
management  and  on  NAVAIR’s  technical  directives  system. 

•  Other  acquisition  workforce  members  provided  examples  of  training  from 
other  federal  agencies  or  commercial  vendors,  such  as  financial  training 
from  the  Graduate  School,  United  States  Department  of  Agriculture,  and 
Management  Concepts,  while  others  said  they  had  brown  bag  lunches  on 
various  topics. 

Figure  2  below  identifies  DOD’s  multifaceted  training  approach,  both 
centralized  and  decentralized.  The  objective  of  the  multifaceted  training, 
in  conjunction  with  the  other  two  certification  components — education 
and  experience —  is  acquisition  personnel  with  the  training,  education,  and 
experience  to  perform  the  acquisition  job. 
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Figure  2:  DOD’s  Multifaceted  Training  Intended  to  Prepare  Acquisition  Personnel  to 
Perform  the  Job 


Objective:  Acquisition  personnel 
with  the  training,  education,  and 
experience  to  perform  the  job 


Source:  GAO  analysis  of  DOD  information  on  acquisition  training. 


Implementation/Evaluation:  As  with  other  programs  and  services  that 
agencies  deliver,  it  is  important  that  agencies  collect  program  performance 
data  during  implementation  and  select  an  analytical  approach  that  best 
measures  the  program’s  effect  to  evaluate  their  training  and  development 
efforts.  DAU  collects  customer  feedback  data  during  implementation  and, 
during  evaluation,  uses  the  four-level  Kirkpatrick  model16  as  an  analytical 
approach  for  measuring  training  effectiveness.  As  a  part  of  evaluating 


16  The  Kirkpatrick  model  is  a  balanced,  multilevel  approach  to  evaluate  an  organization’s 
training  and  development  efforts.  In  this  model,  the  first  level  measures  the  training 
participants’  reaction  to,  and  satisfaction  with,  the  training  program.  The  second  level 
measures  the  extent  to  which  learning  has  occurred  because  of  the  training  effort.  The 
third  level  measures  the  application  of  this  learning  to  the  work  environment  through 
changes  in  behavior  that  trainees  exhibit  on  the  job  because  of  training.  The  fourth  level 
measures  the  impact  of  the  training  program  on  the  agency’s  program  or  organizational 
results. 
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training,  DAU  conducts  student  end-of-course  surveys  (Level  1-Reaction) 
and,  to  a  lesser  degree,  follow-up  surveys  of  students  and  their  managers 
60  and  120  days,  respectively,  after  course  completion  (Level  3-Behavior). 
DAU  tracks  the  scores  from  the  various  surveys  by  survey  section,  such  as 
job  impact  ,  and  uses  red-yellow-green  stoplight  indicators  to  identify  areas 
of  concern  overall  and  by  specific  courses.  DAU  also  administers  pre-  and 
post-training  tests  to  measure  learning  (Level  2-Learning).  To  measure 
organizational  impact  (Level  4-Business  Results),  DAU  employs  measures 
of  efficiency  in  evaluating  and  analyzing  multiyear  data,  such  as  number  of 
students  completing  courses,  cost  efficiency,  and  customer  satisfaction 
trends.  Level  4  assessments  are  resource  intensive  and  have  not  been 
extensively  used  by  DAU. 


Some  Attributes  of  DOD’s 
Certification  Training 
Programs  are  Inadequate 


DOD  is  deficient  in  two  attributes  of  an  effective  training  program — 
determining  the  skills  and  competencies  of  its  workforce  for  planning  and 
front-end  analysis  and  using  performance  data  to  assess  the  results 
achieved  through  training  efforts  during  evaluation.  In  March  2009,  we 
reported  that  USD(AT&L)  lacks  complete  information  on  the  skill  sets  of 
the  current  acquisition  workforce  and  whether  these  skill  sets  are 
sufficient  to  accomplish  DOD’s  missions.17  We  recommended  and  DOD 
agreed  to  identify  and  update  on  an  ongoing  basis  the  number  and  skill 
sets  of  the  total  acquisition  workforce — including  civilian,  military,  and 
contractor  personnel — that  the  department  needs  to  fulfill  its  mission. 
Complete  data  on  skill  sets  are  needed  to  accurately  identify  its  workforce 
gaps.  Not  having  these  data  limits  DOD’s  ability  to  make  informed 
workforce  allocation  decisions.  We  reported  that  USD(AT&L)  was 
conducting  a  competency  assessment  to  identify  the  skill  sets  of  its 
current  acquisition  workforce,  but  also  found  that  the  lack  of  key  data  on 
the  in-house  acquisition  workforce  identified  in  the  prior  report  still  exists, 
though  progress  has  been  made.  Since  we  released  that  report,  DOD 
issued  its  Strategic  Human  Capital  Plan  Update  in  April  2010.  According 
to  DOD,  progress  was  made  in  completing  over  22,000  assessments 
involving  3  of  the  15  career  fields — program  management,  life-cycle 
logistics,  and  contracting  career  fields.  The  assessments  completed  to  date 
represent  approximately  one-fifth  of  the  personnel  and  career  fields. 


17  GAO,  Department  of  Defense:  Additional  Actions  and  Data  Are  Needed  to  Effectively 
Manage  and  Oversee  DOD’s  Acquisition  Workforce,  GAO-09-342  (Washington,  D.C.: 
Mar.  25,  2009). 
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Although  DAU  uses  performance  data — including  customer  feedback, 
number  of  students  completing  classes,  and  cost — to  assess  the  results 
achieved  through  training  efforts  during  evaluation,  USD(AT&L)  has  only 
partially  established  metrics  required  in  2005  by  its  own  guidance  to 
provide  senior  leaders  with  appropriate  oversight  and  accountability  for 
management  and  career  development  of  the  acquisition  workforce.18  The 
purpose  of  these  metrics  is  to  help  DOD  ensure  a  sufficient  pool  of  highly 
qualified  individuals  for  acquisition  positions  and,  therefore,  relates  to  the 
knowledge,  skills,  abilities,  and  size  of  the  acquisition  workforce,  while  the 
DAU  performance  data  measure  the  performance  of  DAU  against  its  goals. 
By  incorporating  these  metrics  into  the  training  and  development 
programs  they  offer,  DOD  can  better  ensure  that  they  adequately  address 
training  objectives  and  thereby  increase  the  likelihood  that  desired 
changes  will  occur  in  the  acquisition  workforce’s  skills,  knowledge, 
abilities,  attitudes,  or  behaviors.  AT&L  programs  lacking  appropriate 
outcome  metrics  will  be  unable  to  demonstrate  how  the  certification 
training  contributes  to  organizational  performance  results.  According  to 
USD(AT&L)’s  Deputy  Director  for  Human  Capital  Initiatives,  DOD  has 
established  some  metrics  to  measure  the  size  of  the  acquisition  workforce 
that  partially  satisfy  the  requirements  identified  in  DOD  Instruction 
5000.66.  For  example,  DOD  measures  the  cumulative  number  of  civilian 
and  military  acquisition  positions  added  as  a  result  of  in-sourcing 
acquisition  functions  performed  by  contractors.19  However,  for  metrics 
related  to  acquisition  workforce  proficiency  and  capability,  there  are  no 
discernable  targets,  except  improvement  over  the  previous  year.  In 
addition,  DOD’s  April  2010  Strategic  Human  Capital  Plan  Update 
identified  an  initiative  to  establish  certification  goals  as  a  management  tool 
for  improving  workforce  quality  by  June  10,  2010.  According  to  the  Deputy 
Director,  certification  goals  are  being  discussed  but  they  had  not  been 
established  at  the  time  of  this  report. 


18  DOD’s  instruction  for  operation  of  the  defense  acquisition,  technology,  and  logistics 
workforce  education,  training,  and  career  development  program  requires  that  appropriate 
metrics  be  developed  for  AT&L’s  senior  leadership  to  have  appropriate  oversight  and 
accountability  for  management  and  career  development  of  the  acquisition  workforce.  DOD 
Instruction  5000.66,  paragraph  E2.5.1.  (Dec.  2005). 

19  In-sourcing  refers  to  converting  functions  performed  by  contractors  to  DOD  personnel. 
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Although  DAU  is  unable  to  provide  all  training  requested  for  acquisition 
workforce  personnel  and  receives  incomplete  data  for  planning  its  training 
schedule,  most  personnel  who  need  required  DAWLA  certification  training 
receive  it  within  required  time  frames.  DAU  plans  the  number  and  location 
of  its  classes  based  on  data  submitted  by  the  Directors  of  Acquisition 
Career  Management  (DACM).  However,  DOD  acquisition  and  training 
officials  noted  that  data  are  generally  incomplete  when  submitted  and 
additional  steps  must  be  taken  during  the  year  to  meet  new  requirements 
as  they  are  identified.  DAU  has  identified  the  need  for  an  integrated 
student  information  system  to  improve  the  quality  of  the  data  and  to 
provide  greater  insight  into  the  workforce  it  supports.  Additionally,  though 
the  number  of  DAU  course  graduates  has  grown  over  the  past  5  years,20 
DAU  has  not  been  able  to  provide  enough  class  seats  to  meet  the  training 
requirements  reported  by  military  departments  and  defense  components. 

DAU  Receives  Incomplete  DAU  receives  annual  DACM  data  submissions  for  the  course  scheduling 
Information  to  Develop  an  process,  but  the  submissions  do  not  provide  the  exact  information  needed 
Annual  Teaching  Schedule  to  determine  training  demand  for  the  acquisition  workforce.  DAU  receives 

class  requirements  data  annually  from  the  DACM  offices  that  it  uses  when 
developing  course  schedules  to  identify  the  number  and  location  of  DAU 
courses.  DACM  offices  compile  this  information  for  all  offices  to  establish 
the  overall  demand  for  each  military  department  and  the  defense  agencies 
for  each  DAU  course.  DAU  and  DACM  offices  work  together  throughout 
the  process  to  improve  the  accuracy  of  this  information  when  possible. 
According  to  DAU  and  DACM  officials,  however,  data  that  are  transmitted 
for  schedule  development  do  not  fully  reflect  all  demand  for  the  upcoming 
year  as  new  requirements  arise  once  the  schedule  is  developed.  As  a  result 
additional  planning  and  coordination  between  DAU  and  DACM  offices  is 
necessary  to  meet  the  training  requirements  of  the  acquisition  workforce. 
For  example,  in  fiscal  year  2009,  DAU  received  requests  for  142  additional 
classes  outside  of  the  normal  scheduling  process.  DAU  was  able  to 
support  45  of  these  requested  classes  in  such  areas  as  program 
management,  contracting,  business  management,  and  logistics.  According 
to  DAU  officials,  resources  for  additional  classes  are  made  available  when 
other  classes  are  cancelled.  Also,  DAU  may  reallocate  allotted  classroom 
seats  among  departments  and  agencies  to  fill  additional  training  needs. 
DAU  officials  stated  that  data  on  selected  acquisition  support  services  that 
are  currently  performed  by  contractors  who  may  transition  to  imhouse 


20  The  term  graduate  refers  to  a  student  who  successfully  completes  a  DAU  class. 


Despite  Incomplete 
Data  and  High 
Demand,  Most 
Workforce  Members 
Receive  Training 
Required  for 
Certification 
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DOD  personnel  are  not  adequate  for  planning  specific  training 
requirements.  Though  DOD  has  established  goals  for  the  number  of 
contracted  personnel  to  be  converted,  DAU  officials  noted  that  the  exact 
time  or  training  backgrounds  of  the  personnel  are  not  known  in  advance. 

DAU  also  uses  acquisition  workforce  data  provided  quarterly  by  the 
DACM  offices  that  include  information  such  as  the  number  of  personnel  in 
each  acquisition  career  field  as  well  as  the  career  level,  job  titles,  and 
status  of  progress  against  certification  requirements  of  each  workforce 
member  to  inform  course  demand  management.  According  to  DAU 
officials,  these  data  provide  a  snapshot  of  the  acquisition  workforce  and 
certification  status,  and  they  use  this  information  to  estimate  the  number, 
location,  and  type  of  classes  needed  by  the  acquisition  workforce  for 
certification.  The  data  are  compiled  to  create  a  demand  management  tool 
that  provides  DAU  with  an  imprecise  estimate  of  course  requirements  and 
are  used  to  supplement  and  inform  the  estimates  developed  during  the 
scheduling  process.  However,  this  demand  management  tool  alone  cannot 
be  used  by  DAU  to  determine  the  exact  number  of  classroom  seats 
required  each  fiscal  year.  According  to  DAU  officials,  the  workforce  data 
collected  may  overstate  training  requirements  because  it  does  not  account 
for  training  that  has  already  been  completed  when  individuals  held  a 
previous  acquisition  position,  nor  does  it  discern  between  multiple  classes 
that  may  fulfill  the  same  training  requirement. 

Citing  incomplete  data  for  scheduling,  as  well  as  other  deficiencies,  DAU 
has  taken  steps  to  procure  a  student  information  system  that  will  improve 
insight  into  and  enhanced  management  of  the  defense  acquisition 
workforce’s  training  needs.  DAU  began  its  market  research  for  an 
integrated  student  information  system  in  December  2007,  viewed  vendor 
presentations  and  demonstrations  throughout  2008,  and  issued  a  request 
for  proposal  in  August  2010.  In  the  request  for  proposal,  DAU  identified 
the  need  for  an  integrated  system  for  registration,  student  services,  career 
management,  schedule  management,  catalog  requirements,  recording 
transcripts,  and  reporting  intended  to  improve  its  management  of  training 
needs  and  schedules.  Without  an  integrated  system,  DAU  states  that  it  will 
remain  reliant  on  a  web  of  decentralized  information  that  makes  reporting 
and  trend  analysis  difficult  and  time-consuming.  A  primary  goal  of  the  new 
system  is  to  provide  a  comprehensive  approach  that  improves,  among 
other  things,  tracking  of  certification  status  and  ensures  training  reaches 
the  right  people  at  the  right  time.  DAU  plans  call  for  the  contractor  to 
complete  implementation  of  this  new  student  information  system 
24  months  after  the  date  of  contract  award,  which  had  not  been  made  as  of 
September  2010. 
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DCAI  develops  its  training  schedule  based  on  the  requirements  expressed 
in  the  individual  development  plans  and  availability  of  DCAI  resources. 
Registration  for  DCAI  courses  is  prescribed  and  based  largely  on  the 
individual  developmental  plans  submitted  by  DCAA’s  approximately  3,700- 
member  auditing  workforce  in  fiscal  year  2009.  Each  year  DCAA 
employees  develop  an  individual  development  plan  that  lists  DCAI  courses 
as  well  as  outside  training  deemed  necessary  with  the  input  and  approval 
of  their  supervisors.  This  information  is  input  into  a  system  that  tracks 
course  requirements  and  individuals’  status  against  training  requirements. 
Individuals  are  automatically  enrolled  into  the  scheduled  DCAI  courses. 


Personnel  Receive 
Required  Training  within 
Required  Time  frames 
According  to  DAU  Data, 
However,  DAU  Cannot 
Support  All  Training 
Requirements  for  the 
Defense  Acquisition 
Workforce 


Most  of  the  acquisition  workforce  receives  training  within  required 
certification  time  frames.  At  the  end  of  fiscal  year  2009,  approximately 
90  percent  of  the  133,103  members  of  the  defense  acquisition  workforce 
had  met  certification  requirements  associated  with  their  position  or  were 
within  allowed  time  frames  to  do  so.  Acquisition  workforce  members  we 
met  with  from  all  three  military  departments  and  the  Defense  Contract 
Management  Agency  (DCMA)  noted  challenges  for  receiving  training  at 
the  time  and  location  they  desired,  noting  that  local  DAU  locations  would 
fill  up  quickly  and  that  they  would  often  have  to  register  for  courses 
multiple  times  prior  to  enrollment.  However,  acquisition  staff  and 
supervisors  told  us  that  this  had  little  effect  on  being  certified  within  the 
required  time  frames  for  their  current  positions.  Nearly  all  of  the 
remaining  uncertified  personnel  required  training  to  become  certified. 
While  additional  training  was  needed,  these  individuals  may  also  have 
been  deficient  in  meeting  education  or  experience  requirements  also 
needed  for  DAWIA  certification.  Furthermore,  DACM  officials  noted  that 
there  could  be  a  number  of  reasons  these  individuals  had  not  received 
required  training  and  stated  that  while  some  individuals  may  not  have 
adequately  planned  for  their  training  needs,  other  factors,  such  as 
deployment  of  military  personnel  abroad,  may  have  limited  their  access  to 
training. 


DCAA  auditors  do  not  face  the  same  issues  with  DAWIA  certification  as 
the  rest  of  the  acquisition  workforce.  According  to  DCAI  officials,  this  is 
largely  because  they  do  not  have  to  coordinate  demand  for  courses  across 
several  different  agencies.  All  new  hires  are  automatically  enrolled  in  the 
courses  required  for  level  I  and  level  II  DAWIA  certification.21  Additionally, 


21  In  fiscal  year  2009,  there  were  2,764  level  I  and  level  II  auditors,  making  up  approximately 
73  percent  of  the  total  DAWIA  auditing  workforce. 
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DOD  reported  that  approximately  99  percent  of  the  auditing  workforce 
had  met  certification  requirements  or  were  within  allowed  time  frames  to 
do  so.  By  completing  the  mandatory  learning  track  taught  through  DCAI 
classes,  DCAA  auditors  complete  certification  training  within  required 
time  frames. 

Even  though  90  percent  of  the  acquisition  personnel  who  required 
certification  training  for  their  current  position  received  training  on  time  or 
were  within  allowed  time  frames  to  do  so,  DAU  acknowledges  that 
requests  for  acquisition  workforce  training  as  a  whole  submitted  by  the 
DOD  components  and  military  departments  exceed  what  DAU  can 
provide.  DAU  has  incorporated  expansion  of  training  into  its  strategic 
plans.  In  its  Strategic  Plan  for  2010-2015,  DAU  notes  that  it  will  play  a  key 
role  in  the  USD(AT&L)  acquisition  workforce  growth  strategy.  For 
example,  USD(AT&L)  efforts  to  grow,  train,  and  develop  the  defense 
acquisition  workforce  will  affect  DAU’s  strategic  planning  over  the  next 
several  years.  DAU  notes  that  workforce  growth  goals  put  forth  by  the 
Secretary  of  Defense  in  April  2009  will  increase  the  demand  for  DAU 
training  and  therefore  affect  how  DAU  plans  for  development  of 
acquisition  personnel,  requiring  careful  consideration  of  resource 
allocation.  The  strategic  plan  also  points  out  a  number  of  other  factors 
that  will  drive  the  demand  for  acquisition  workforce  training  in  the  coming 
years,  including  annual  workforce  turnover,  turnover  related  to  Base 
Realignment  and  Closure,  and  support  for  new  acquisition  development 
needs.  As  part  of  its  strategy,  DAU  has  also  established  short-term  goals  to 
expand  training  capacity  in  its  fiscal  year  2010  Organizational  Performance 
Plan,  including  expanding  classroom  training  by  10,000  seats  over  fiscal 
year  2009  levels.  DAU  officials  stated  that  they  plan  to  increase  capacity 
further  to  provide  54,000  classroom  seats  in  fiscal  year  2011.  In  addition, 
DAU  established  and  has  fulfilled  a  strategic  goal  of  graduating  150,000 
students  from  its  Web-based  courses  annually. 

DAU  has  increased  the  total  number  of  course  graduates  and  classes  in 
recent  years  to  address  demand  for  acquisition  training.  DAU  has 
supported  more  classes  than  in  the  past,  seeing  an  increase  from  1,279 
classroom  courses  in  fiscal  year  2005  to  1,505  in  fiscal  year  2009.  In 
addition,  from  fiscal  year  2005  through  2009,  the  number  of  individual 
graduates  from  DAU  classroom  and  Web-based  courses  rose  by 
approximately  77  percent  (see  fig.  3). 
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Figure  3:  DAU  Graduates,  Fiscal  Year  2005-2009 
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Source:  AT&L  Data  Mart. 


To  support  increases  in  certification  training  demand  due  to  workforce 
growth  through  new  hiring  and  in-sourcing,  DAU  uses  funding  from  the 
Defense  Acquisition  Workforce  Development  Fund  to  provide  additional 
facilities  and  courses.22  Though  the  majority  of  funding  is  intended  to 
support  the  hiring  of  new  staff,  DAU,  military  departments,  and  defense 
agencies  received  more  than  $225  million  dollars  to  support  new  training 
and  additional  seats  in  fiscal  years  2008  and  2009.  Funds  have  been  used 
by  the  military  departments  to  support  Army  and  Navy  acquisition  boot 
camps,23  the  Air  Force’s  mission-ready  contracting  course,  and  other 
acquisition  training  developed  by  specific  military  commands.  For 


“2  The  National  Defense  Authorization  Act  for  fiscal  year  2008  established  the  Defense 
Acquisition  Workforce  Development  Fund  to  provide  additional  funds  for  the  recruitment, 
training,  and  retention  of  acquisition  personnel  of  the  Department  of  Defense. 

Pub.  L.  No.  110-181  §  852;  codified  in  10  U.S.C.  §  1705. 

23  Boot  camps  provide  an  overview  of  acquisition  and  contracting  issues  developed  by 
commands. 
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example,  funding  was  used  to  develop  and  implement  the  “jump  start” 
program  at  the  Air  Force’s  Aeronautical  Systems  Center  that  combines 
material  taught  through  DAU’s  Level  I  contracting  courses  with  Air  Force- 
specific  information.  The  Defense  Acquisition  Workforce  Development 
Fund  has  also  been  used  by  DAU  to  expand  its  teaching  facilities,  hire 
additional  instructors,  and  schedule  additional  classes  needed  for  DAWIA 
certification.  DAU  received  nearly  $165  million  in  fiscal  years  2008  and 
2009  to  expand  training.  In  fiscal  year  2009,  this  funding  permitted  DAU  to 
offer  nearly  7,000  additional  classroom  seats  in  31  different  courses.  DAU 
has  also  used  these  funds  to  develop  new  training — such  as  a  4-week 
course  focusing  on  the  Federal  Acquisition  Regulation  that  senior  DOD 
contracting  officials  said  was  needed  to  provide  a  foundation  for 
acquisition  fundamentals — and  to  support  acquisition  professionals  in  the 
field  through  Service  Acquisition  Workshops  and  expanded  contingency 
acquisition  training. 

Despite  these  increased  class  offerings  that  have  accommodated  more 
graduates,  DAU  has  not  been  able  to  provide  the  total  number  of 
classroom  seats  that  are  requested  by  the  defense  acquisition  workforce 
through  the  DACMs.  Classroom  seats  requested  and  class  seats  scheduled 
both  increased  from  fiscal  year  2007  through  2009.  For  example,  in  fiscal 
year  2009,  DOD  components  requested  52,998  seats  for  the  acquisition 
workforce  across  66  different  DAU  classroom  courses;  DAU  was  able  to 
allocate  resources  to  meet  71  percent  of  this  demand  based  on  its  annual 
budget.  However,  DAU  made  use  of  the  Defense  Acquisition  Workforce 
Development  Fund  to  provide  additional  classroom  seats  to  meet  the 
demand  for  training,  allowing  them  to  meet  87  percent  of  the  workforce’s 
requirement  in  fiscal  year  2009.  Further,  DAU  data  demonstrate  that 
workforce  personnel  who  require  certification  training  for  their  current  or 
future  position  within  their  career  field  constitute  a  large  majority  of 
classroom  students  graduating  from  DAU  courses. 
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DOD  Has  Addressed 
Most  Previous 
Training 

Recommendations, 
However,  Some 
Remain  to  Be 
Implemented 


DOD  reports  that  most  of  the  training-related  recommendations  from 
previous  reviews — the  Gansler  Commission,  the  Panel  on  Contracting 
Integrity,  and  our  prior  report — have  been  fully  implemented.  We  reviewed 
19  recommendations  addressing  some  aspect  of  acquisition  training  and 
found  that  11  have  been  fully  implemented,  4  have  been  partially 
implemented,  and  4  have  not  been  implemented  but  action  has  been  taken. 

Two  of  the  four  Gansler  Commission  Report  recommendations  have  been 
implemented;  however,  the  Army  and  Office  of  the  Secretary  of  Defense 
(OSD)  need  to  take  additional  steps  to  ensure  the  Army  “trains  as  it  fights” 
and  that  DAU  has  the  resources  it  needs  to  train  the  acquisition  workforce. 
Nine  of  the  11  Panel  on  Contracting  Integrity  recommendations  have  been 
fully  implemented.  DOD  has  taken  actions  to  address  performance-based 
acquisitions  training;  however,  DOD  has  not  conducted  a  formal 
assessment  of  its  guidance  or  the  training.  Also,  on  the  basis  of 
information  from  DOD,  we  could  not  determine  whether  it  conducted  a 
review  of  its  Fraud  Indicator  Training  and  the  Continuity 
Book/Contracting  Office  Transition  Plan. 

One  of  the  training-related  recommendations  we  made  to  DCAA  has  been 
partially  implemented,  and  three  have  not  been  implemented  but  action 
was  taken.  DCAA  needs  to  take  further  steps  to  develop  appropriate 
training  for  its  auditors  and  it  should  seek  outside  expertise  in  doing  so. 


Army  and  OSD  Have  Taken 
Steps  to  Respond  to 
Gansler  Commission 
Training  Recommended 
Changes,  but  Additional 
Steps  Need  to  Be  Taken 


In  response  to  the  Gansler  Commission  report,  the  Army  and  OSD  have 
taken  steps  to  improve  training  and  implement  the  report’s 
recommendations.  In  2007  the  Gansler  Commission  made  4  overarching 
recommendations  and,  within  those  4,  the  Commission  described  35  more 
in-depth  recommendations  on  Army  acquisition  and  program  management 
in  expeditionary  operations.  Four  of  those  in-depth  recommendations 
pertain  to  training  the  DAWIA  workforce.24  As  shown  in  table  3,  2  of  the 
commission’s  training  recommendations  have  been  fully  implemented, 
while  the  remaining  2  training  recommendations  require  additional  action. 


24  The  report  made  five  additional  recommendations  within  the  overarching  training 
recommendation.  The  fifth  recommendation  pertains  to  training  for  Contracting  Officer’s 
Representatives,  which  are  not  included  in  the  DAWIA  workforce  and,  therefore,  the 
recommendation  was  not  included  in  this  report.  Training  recommendations  related  to 
DOD  personnel  in  a  contingency  environment  with  acquisition  responsibilities  but  who  are 
not  covered  by  DAWIA  will  be  reviewed  in  a  future  GAO  report. 
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Table  3:  Implementation  Status  of  Gansler  Commission  Training  Recommendations 

Recommendations 

DOD  status3 

Our  assessment 

'T rain  as  we  fight:"  exercise  for  rapid  acquisitions,  logistics,  and  contracting  in  expeditionary 
operations.11 

Completed 

Partially 

implemented 

Develop  and  field  the  contract  tools  needed  for  the  expeditionary  forces. 

Completed 

Fully  implemented 

Focus  DAU  to  train  and  educate  the  civilian  and  military  acquisition,  logistics,  and 
contracting  workforce  for  expeditionary  operations. 

Completed 

Fully  implemented 

Provide  DAU  the  necessary  resources  to  accommodate  the  Army's  emphasis  on  Level  1 
certification. 

Completed 

Partially 

implemented 

Source:  GAO  analysis  and  Gansler  Commission  Report. 


The  agency  status  reported  in  this  column  is  taken  from  the  OS  D  and  Army  reports  to  Congress, 
which  used  different  language  to  report  the  status  of  implementation. 

The  recommendations'  language  has  been  edited;  the  full  language  can  be  found  in  Report  of  the 
Commission  on  Army  and  Program  Management  in  Expeditionary  Operations  -  Urgent  Reform 
Required  ("Gansler  Commission  Report”)  (Washington,  D.C.:  Oct.  31,  2007). 

While  DOD  has  taken  action,  additional  steps  are  needed  to  fully 
implement  the  Gansler  Commission  training  recommendations.  The 
following  is  our  rationale  for  ongoing  efforts  needed  to  continue  for  the 
Army  and  OSD  to  fully  address  the  Gansler  Commission  training 
recommendations. 

•  “Train  as  we  fight:”  DOD  officials  stated  that  training  exercises  include 
contracting  and  logistics,  incorporate  lessons  learned,  and  may  include 
training  for  commanders,26  but  we  could  not  determine  the  extent  to  which 
they  are  included  due  to  lack  of  documentation.  The  Army  has 
mechanisms  to  capture  lessons  learned,  but  it  is  unclear  how  they  are 
incorporated  into  training  exercises.  For  example,  the  Expeditionary 
Contracting  Command  informally  receives  lessons  learned  from  other 
Army  commands  and  brigades,  but  we  could  not  determine  whether  and 
how  they  are  incorporated  into  training  exercises  because  they  are  not 
tracked  or  formally  documented. 

•  Provide  DAU  with  needed  resources  to  certify  Army  individuals  requiring 
level  I  certification:  DAU  and  the  Army  do  not  have  the  needed  resources 
to  emphasize  level  I  DAWIA  certification,  according  to  DOD  officials.  DAU 
is  not  adequately  funded  to  meet  the  acquisition  training  demand  DOD- 
wide.  For  example,  according  to  OSD  officials,  DAU  is  not  fully  funded  to 


35  The  Army  also  developed  Operational  Contract  Support  pre-deployment  training  to 
implement  this  recommendation.  However,  it  is  not  required  for  operational  commanders 
and  Army  officials  could  not  tell  us  how  many  opted  to  complete  the  1-hour  training.  This 
training  will  also  be  reviewed  in  a  future  GAO  report. 
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meet  the  fiscal  year  2011  services  and  defense-wide  agency  demand  for 
contracting  level  I  courses.  DAU  projects  meeting  60  percent  of  the  fiscal 
year  2011  requested  seats  for  these  level  I  courses.  The  Army  depends  not 
only  on  DAU,  but  also  on  the  Army  Logistics  University  and  the  Air  Force 
Mission  Ready  Airman  Course  to  provide  the  contracting  training  needed 
to  its  active  component  personnel. 


DOD  Has  Implemented 
Most  Training 
Recommendations  from 
the  Panel  on  Contracting 
Integrity 


DOD  has  not  implemented  all  recommended  actions  related  to  defense 
acquisition  workforce  training  included  in  the  Panel  on  Contracting 
Integrity’s  2008  and  2009  reports  to  Congress.  The  Panel  recommended  a 
total  of  49  actions  to  improve  acquisition  outcomes.  Of  these 
recommended  actions,  11  specifically  addressed  acquisition  training.  See 
table  4  for  a  complete  list  of  the  recommended  actions  related  to  training 
included  in  the  Panel’s  reports  to  Congress  in  2008  and  2009. 


Table  4:  Panel  2008  and  2009  Recommended  Training  Actions’  Status 

2009  Recommended  actions 

Panel  reported 
status 

Our  assessment 

Use  case  studies  in  contracting  integrity  to  promote  discussion  and  communicate 
standards  in  areas  of  ambiguity. 

Completed 

Fully  implemented 

Have  Defense  Procurement  and  Acquisition  Policy  and  senior  contracting  leaders 
develop  short-term  gap  closure  strategies  for  recruiting,  hiring,  development,  and 
retention  initiatives  for  consideration  of  the  Defense  Acquisition  Workforce 

Development  Fund. 

Completed 

Fully  implemented 

Assess  effectiveness  of  departmental  guidance  and  training  for  executing  performance- 
based  acquisition  and  perform  gap  analysis  in  conjunction  with  DAU. 

Completed 

Not  implemented- 
action  taken 

Provide  updated  guidance  and  training  on  competition  initiatives  and  continue 
emphasis  on  enhancing  competition  for  contracts  and  orders  placed  under  multiple- 
award  contracts. 

Completed 

Fully  implemented 

Update  and  Web-enable  the  J  oint  Contingency  Contracting  Handbook;  build  upon 
previous  efforts  and  revise  contingency  contracting  training  curriculum  as  required. 

Completed 

Fully  implemented 

Communicate  with  contracting  officers,  auditors,  and  DCMA  representatives  regarding 
an  advanced  course  on  procurement  fraud  indicators  and  determine  feasibility  of 
development  during  2009. 

Completed 

Fully  implemented 

2008  Recommended  actions 

Assess  need  for  revised/additional  training  on  competition  requirements  and  differing 
pricing  alternatives. 

Completed 

Fully  implemented 

Improve  training  by  leveraging  Marine  Corps  and  Air  Force  training  capabilities. 

Completed 

Fully  implemented 

Improve  training  on  how  to  run  a  contracting  office  in  a  combat/contingent  environment. 

Completed 

Fully  implemented 

Subgroups  review  Fraud  Indicator  Training  and  Continuity  Book/Contracting  Office 
Transition  Plan. 

Completed 

Partially  implemented 

Create  DAU  Training  Module  on  Procurement  Fraud  Indicators  and  Risk  Mitigation. 

Completed 

Fully  implemented 

Source:  GAO  analysis  and  Panel  on  Contracting  Integrity  2008  and  2009  reports. 
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While  the  Panel  reported  that  all  of  the  recommended  actions  had  been 
completed,  we  determined  that  two  of  the  recommended  actions 
pertaining  to  training  had  not  been  fully  implemented;  we  determined  that 
one  was  not  implemented,  but  action  was  taken,  and  one  has  been 
partially  implemented. 

•  Assess  effectiveness  of  DOD  guidance  and  training  for  executing 
performance-based  acquisition  and  perform  gap  analysis  in  conjunction 
with  DAU:  The  report  did  not  indicate  if  DOD  conducted  a  formal 
assessment  of  departmental  guidance  or  a  gap  analysis  of  training.  The 
Panel’s  Appropriate  Contracting  Approaches  &  Techniques  Subcommittee 
worked  with  DAU  to  determine  if  training  needed  to  be  updated  and 
collected  examples  of  complex  and  high-dollar  acquisitions  and  posted 
them  to  an  Acquisition  Community  Connection  Web  site.  The  report  also 
noted  that  DAU  would  select  the  best  examples  from  this  group  for 
inclusion  in  its  web-based  integrated  training  tool. 

•  Review  Fraud  Indicator  Training  and  Continuity  Book/Contracting  Office 
Transition  Plan:  The  Panel  report  did  not  specifically  address  whether  a 
formal  review  determined  specific  gaps  in  training,  as  recommended.  In 
2008,  the  Panel’s  Contracting  Integrity  in  a  Combat/Contingent 
Environment  Subcommittee  reported  that  DOD  incorporated  transition 
planning  and  fraud  indicator  training  into  the  Joint  Contingency 
Contracting  Handbook  and  updated  DAU’s  Joint  Contingency  Contracting 
Course. 

In  addition  to  the  recommendations  above  that  are  specific  to  training,  the 
Panel  recommended  other  actions  that  also  affected  training,  one  of  which 
was  not  fully  implemented.  The  Contractor  Employee  Conflicts  of  Interest 
Subcommittee  reviewed  and  recommended  that  the  Secretary  of  Defense 
issue  guidance  to  clarify  the  circumstances  in  which  contracts  risk 
becoming  improper  personal  services  contracts.26  DOD  formed  an  ad  hoc 
team  to  respond  to  the  recommendation  which  focused  on  establishing  a 
Defense  Federal  Acquisition  Regulation  Supplemental  case,  DAU  course 
updates,  and  a  DOD  instruction  update.  While  the  DOD  instruction  was 
published,  the  Panel’s  report  did  not  mention  the  status  of  the  Defense 
Federal  Acquisition  Regulation  Supplemental  case  or  the  DAU  course 
updates. 


26  GAO,  Defense  Contracting:  Army  Case  Study  Delineates  Concerns  with  Use  of 
Contractors  as  Contract  Specialists,  GAO-08-360  (Washington,  D.C.:  Mar.  26,  2008). 
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DCAA  Has  Not  Fully 
Implemented  GAO’s 
Recommendations  on 
Auditor  Training 


In  2009,  we  made  four  recommendations  regarding  DCAA  auditor  training, 
which  have  not  been  fully  implemented  (see  table  5  for  our  full 
recommendations).  Three  of  the  recommendations  have  not  been 
implemented  but  action  was  taken,  and  one  has  been  partially 
implemented.  As  stated  in  our  September  2009  report,  DCAA  faces  many 
challenges  and  fundamental  structural  and  cultural  changes  related  to 
developing  a  strong  management  environment  and  human  capital  strategic 
plan.  First,  we  recommended  that  once  DCAA  establishes  a  risk-based 
audit  approach  it  will  need  to  develop  a  staffing  plan  that  identifies  auditor 
resource  requirements  including  training  needs.  Second,  we  recommended 
that  DCAA  establish  a  position  for  an  expert  or  consult  with  an  outside 
expert  on  auditing  standards  to  shape  audit  policy,  provide  guidance,  and 
develop  training.  While  DCAA  has  taken  steps  to  improve  its  audit  training, 
such  as  implementing  an  initiative  to  identify  the  knowledge,  skills,  and 
competencies  required  for  DCAA  auditors  and  develop  training,  according 
to  a  DCAA  official,  it  has  not  yet  hired  or  consulted  with  an  outside  expert 
on  auditing  to  shape  its  policies  and  provide  guidance.  Third,  we 
recommended  that  DCAA  develop  agencywide  training  on  government 
audit  standards.  Agency  officials  stated  that  as  of  July  2010,  DCAA  had 
developed  a  new  online,  introductory  course  on  Generally  Accepted 
Government  Auditing  Standards  (GAGAS)  all  DCAA  auditors  are  supposed 
to  complete  by  September  30,  2010.  We  are  reviewing  the  new  course 
content  and  continue  to  work  with  DCAA  on  planned  improvements  to 
address  the  fundamental  structural  and  cultural  changes  previously 
identified.  Fourth,  as  DCAA’s  audit  quality  assurance  program  identifies 
actions  needed  to  address  serious  deficiencies  and  GAGAS 
noncompliance,  we  recommended  that  DCAA  provide  training  and  follow¬ 
up  to  ensure  that  appropriate  corrective  actions  have  been  taken.  DCAA 
has  issued  audit  alerts  and  provided  some  guidance  through  periodic 
regional  office  and  field  office  conferences,  but  has  not  yet  incorporated 
this  guidance  into  the  body  of  its  DCAI  audit  courses. 
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Table  5:  GAO  Training  Recommendations’  (for  DCAA)  Status 

Recommendation 

DCAA  status 

Our  assessment 

Based  on  a  risk-based  audit  approach,  develop  a  staffing  plan  that  identifies  auditor 
resource  requirements  as  well  as  auditor  skill  levels  and  training  needs. 

NAa 

Not  implemented-action 
taken 

Establish  a  position  for  an  expert  on  auditing  standards  or  consult  with  an  outside  expert 
on  auditing  standards  to  assist  in  revising  contract  audit  policy,  providing  guidance  on 
sampling  and  testing,  and  developing  training  on  professional  auditing  standards. 

NA 

Not  implemented-action 
taken 

Develop  agencywide  training  on  government  audit  standards.  This  training  should 
emphasize  the  level  of  assurance  intended  by  the  various  types  of  engagements  and 
provide  detailed  guidance  on  auditor  independence,  planning,  fraud  risk,  level  of  testing, 
supervision,  auditor  judgment,  audit  documentation,  and  reporting. 

NA 

Not  implemented-action 
taken 

Make  appropriate  recommendations  to  address  annual  quality  assurance  review  findings 
of  serious  deficiencies  and  GAGAS  noncompliance,  provide  training,  and  followup  to 
ensure  that  appropriate  corrective  actions  have  been  taken. 

NA 

Partially  implemented 

Source:  GAO  analysis  and  GAO-09-468. 

aGAO-09-468  was  published  in  September  2009.  Officials  from  DCAA  stated  thatthey  are  currently 
reviewing  policy  to  implement  these  recommendations. 

Conclusions 


DOD’s  acquisition  workforce  training  program  demonstrates  many 
attributes  of  effective  training  and  development  programs;  however,  there 
is  room  for  further  improvement.  DOD  recognizes  the  need  to  continue  its 
efforts  to  assess  competencies  for  its  acquisition  workforce.  Importantly, 


if  this  effort  is  not  completed,  DOD  will  be  limited  in  its  ability  to  identify 
gaps  in  the  skill  sets  of  acquisition  personnel,  ultimately  hampering  its 
ability  to  effectively  acquire  the  goods  and  services  it  needs  to  accomplish 
its  mission.  Notably,  opportunities  exist  to  improve  the  measurement  of 
training’s  impact  on  overall  organizational  performance.  If  DOD  is  to  fully 
assess  performance  improvements,  it  needs  to  go  beyond  measuring  the 


size  of  the  workforce.  To  provide  appropriate  oversight  of  the  proficiency 
and  capability  of  its  acquisition  workforce,  DOD  will  need  metrics  to 
measure  skills,  knowledge,  and  abilities,  and  how  certification  training 
contributes  to  organizational  performance  results.  Furthermore,  DAU 
faces  challenges  with  the  management  and  forecasting  of  training  demand 
data  for  specific  training  courses,  which  hinders  its  ability  to  accurately 


plan  the  course  schedule  for  the  upcoming  year  in  a  manner  that  will 
facilitate  getting  the  required  training  to  acquisition  workforce  members  in 
an  efficient  and  cost-effective  manner.  Accurate  and  timely  forecasting  of 
acquisition  workforce  training  requirements  and  the  development  of 
metrics  for  the  proficiency  of  the  workforce  are  imperative  to  support 
DOD’s  initiatives  to  improve  and  grow  the  acquisition  workforce. 
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Recommendations 
Executive  Action 


for  recommen(l  that  the  Secretary  of  Defense  direct  USD(AT&L)  to  take 
the  following  two  actions  to  improve  the  development,  implementation, 
and  evaluation  of  acquisition  workforce  training. 


•  In  order  to  demonstrate  and  track  how  training  efforts  contribute  to 
improved  acquisition  workforce  performance,  establish  milestones  for  the 
development  of  metrics  to  measure  how  acquisition  certification  training 
improves  the  proficiency  and  capability  of  the  acquisition  workforce. 

•  In  order  to  improve  DOD’s  ability  to  identify  specific  acquisition  training 
needs  for  planning  and  front-end  analysis,  establish  a  time  frame  for 
completion  and  ensure  resources  are  available  for  implementing  an 
enterprisewide,  integrated  student  information  system. 


Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  this  report  to  DOD  for  comment.  In  written 
comments,  DOD  did  not  agree  with  our  first  recommendation  and  did 
agree  with  our  second  recommendation.  DOD’s  comments  are  discussed 
below  and  are  reprinted  in  appendix  III. 

DOD  did  not  concur  with  our  recommendation  that  it  should  develop 
milestones  for  the  development  of  metrics  to  demonstrate  and  track  how 
acquisition  certification  training  improves  the  acquisition  workforce 
performance.  While  DOD  agreed  that  metrics  should  be  used  to  measure 
the  capability  of  the  acquisition  workforce,  it  believes  developing 
milestones  for  such  metrics  is  unnecessary  because  existing  metrics  can 
be  used  to  this  end.  DOD  states  that  workforce  capability  is  a  function  of 
having  the  correct  number  of  people  working  in  the  right  areas  with  the 
proper  level  of  education,  training,  and  experience.  Specifically,  DOD 
notes  five  metrics  used  to  measure  size  and  composition  of  the  workforce 
as  well  as  the  education,  training,  and  experience  levels  of  the  individuals 
that  comprise  it. 

We  recognize  that  metrics  for  measuring  these  elements  are  valuable  for 
gaining  insight  into  the  degree  to  which  required  workforce  personnel  are 
being  certified  and  filling  needed  positions.  However,  as  we  note  in  this 
report  and  in  GAO’s  guidance  for  assessing  strategic  training  and 
development  programs,  training  effectiveness  must  be  measured  against 
organizational  performance.  DOD’s  existing  metrics  measure  the  outputs 
for  certification  training,  not  the  outcome  in  terms  of  proficiency  or 
capability  of  the  acquisition  workforce.  Without  outcome  metrics,  DOD 
cannot  demonstrate  how  certification  training  contributes  to  improving 
organizational  performance  results.  Given  the  scale  and  value  of  DOD 
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acquisitions,  we  maintain  that  metrics  that  link  training  to  acquisition 
performance  outcomes  should  be  developed  by  the  department. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Defense,  the  DOD 
Inspector  General,  and  other  interested  parties.  The  report  is  also  available 
at  no  charge  on  the  GAO  Web  site  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  contact  me 
at  (202)  512-4841  or  needhamjkl@gao.gov.  Contact  points  for  our  Offices 
of  Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last 
page  of  this  report.  GAO  staff  who  made  major  contributions  to  this  report 
are  listed  in  appendix  IV. 


John  K.  Needham 

Director,  Acquisition  and  Sourcing  Management 
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List  of  Congressional  Committees 

The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  McCain 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Daniel  Inouye 
Chairman 

The  Honorable  Thad  Cochran 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Ike  Skelton 
Chairman 

The  Honorable  Howard  P.  McKeon 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Norman  D.  Dicks 
Chairman 

The  Honorable  C.W.  Bill  Young 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I:  Scope  and  Methodology 


Congress  included  a  provision  in  the  National  Defense  Authorization  Act 
for  Fiscal  Year  2010  requiring  us  to  report  on  the  efficacy  of  the 
Department  of  Defense’s  (DOD)  acquisition  and  audit  workforce  training.1 
To  determine  the  efficacy  of  DOD’s  acquisition  and  audit  workforce 
training,  we  assessed  (1)  DOD’s  capability  to  provide  defense  acquisition 
workforce  certification  training,  (2)  the  extent  that  such  training  reaches 
members  of  DOD’s  acquisition  workforce,  and  (3)  the  extent  that  training 
recommendations  from  previous  reviews,  including  the  Gansler 
Commission,  have  been  implemented.  We  were  not  able  to  report  on  the 
efficacy  of  training  for  the  Defense  Contract  Audit  Agency’s  (DCAA) 
auditing  career  field  because  DCAA  lacks  a  strategic  plan.  A  strategic  plan 
is  a  key  document  for  assessing  training  programs  using  the  strategic 
training  efforts  attributes. 

For  this  engagement,  we  focused  on  training  for  DOD  personnel  covered 
under  the  Defense  Acquisition  Workforce  Improvement  Act  (DAWLA).2  To 
assess  DOD’s  capability  to  provide  defense  acquisition  workforce 
certification  training,  we  compared  DOD’s  certification  training  programs 
and  processes  with  the  attributes  of  effective  training  and  development 
programs  identified  in  GAO’s  2004  guide  for  assessing  strategic  training 
and  development  efforts  in  the  federal  government,  which  we  identified  as 
the  most  comprehensive  source  for  attributes  of  effective  training 
programs  for  our  purpose.3  We  interviewed  officials  at  the  Defense 
Acquisition  University  (DAU)  and  Defense  Contract  Audit  Institute  (DCAI) 
to  obtain  an  understanding  of  their  training  programs  and  processes,  and 
we  obtained  documents — such  as  briefings,  guidance,  strategic  plans,  and 
course  catalogs — describing  the  training  programs  and  processes.  We 


1  Pub.  L.  No.  111-84,  §  1108(b)(2)  (2009). 

2  Pub.  L.  No.  101-510,  §  1202(a)  (1990).  For  the  purposes  of  this  report,  the  personnel 
covered  under  DAWIA  will  be  referred  to  as  the  acquisition  workforce.  GAO  will  review 
training  for  DOD  personnel  in  contingency  and  non-contingency  environments  who  have 
acquisition  responsibilities  but  are  not  covered  by  DAWLA  in  two  forthcoming  reports. 

3  GAO,  Human  Capital:  A  Guide  for  Assessing  Strategic  Training  and  Development 
Efforts  in  the  Federal  Government,  GAO-04-546G  (Washington,  D.C.:  Mar.  2004).  We 
compared  DOD’s  certification  training  program  for  14  of  the  15  career  fields  with  the 
attributes  of  effective  training  and  development  programs  identified  in  GAO’s  guide  for 
assessing  strategic  training.  However,  we  were  not  able  to  apply  these  criteria  for  the 
fifteenth  career  field — auditing — because  DCAA  does  not  have  a  strategic  plan  in  place 
(GAO,  DCAA  Audits:  Widespread  Problems  with  Audit  Quality  Require  Significant 
Reform. ,  GAO-09-468  (Washington,  D.C.:  Sept.  2009)).  Because  a  strategic  plan  is  a  key 
document  for  assessing  training  programs  using  the  strategic  training  efforts  criteria,  we 
did  not  assess  the  attributes  of  training  for  the  auditing  career  field. 
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interviewed  the  Directors  of  Acquisition  Career  Management  (DACM)  for 
the  military  services  and  defense  agencies  to  obtain  an  understanding  of 
their  role  in  DOD  training,  to  obtain  their  views  on  the  effectiveness  and 
usefulness  of  DAU  training,  and  to  find  out  whether  supplementary 
training  is  provided  by  the  military  services.  We  interviewed  the  leaders  of 
the  functional  integrated  process  teams  that  support  the  functional  leaders 
of  the  15  DAWLA  career  fields  to  obtain  an  understanding  of  their  role  in 
Acquisition,  Technology,  and  Logistics’  (AT&L)  process  and  criteria  for 
reviewing  and  approving  acquisition  workforce  training.  In  addition,  we 
visited  selected  military  commands  and  program  offices  within  those 
commands  to  obtain  customer  perspectives  on  the  effectiveness  and 
usefulness  of  DAU  training  and  to  determine  the  use  of  supplementary 
training.  For  this  purpose,  we  selected  a  nongeneralizable  sample  of  one 
command  from  each  military  service  based  on  the  following  criteria:  (1) 
high  level  of  procurement  dollars  spent  in  fiscal  years  2008  and  2009 
relative  to  other  commands  in  their  military  service,  based  on  data  from 
the  Federal  Procurement  Data  System-Next  Generation;  (2)  large  number 
of  DAU  courses  completed  in  fiscal  years  2008-2009;  and  (3)  proximity  to  a 
DAU  regional  office  with  an  on-site  dean.  The  commands  we  visited  were 
the  Army  Aviation  and  Missile  Command  (AMCOM)  in  Huntsville, 
Alabama;4  the  Air  Force  Materiel  Command  (AFMC)  in  Dayton,  Ohio;  and 
the  Naval  Air  Systems  Command  (NAVAIR)  at  Patuxent  River,  Maryland. 

In  selecting  program  offices  to  visit,  we  reviewed  our  assessment  of 
selected  weapon  programs5  and  consulted  with  the  GAO  team  responsible 
for  our  assessment  to  determine  which  program  offices  would  likely  have 
a  large  cross-section  of  acquisition  workforce  personnel  with  whom  to 
discuss  training.  We  visited  the  following  program  offices:  Joint  Attack 
Munition  Systems  and  Apache  at  AMCOM;  Broad  Area  Maritime 
Surveillance  Unmanned  Aircraft  System  and  E-2D  Advance  Hawkeye  at 
NAVAIR;  and  Global  Hawk  Unmanned  Aircraft  System  at  AFMC.  At  AFMC, 
we  also  visited  the  Aeronautical  Systems  Center’s  Contracting  Directorate, 
and,  at  AMCOM,  we  visited  the  Contracting  Center.  We  also  visited 
Defense  Contract  Management  Agency  personnel  to  obtain  their 
perspectives  on  DAU  training  and  to  find  out  about  their  use  of 
supplementary  training.  Finally,  we  visited  a  nongeneralizable  sample  of 
two  DCAA  locations — the  Alabama  Branch  Office  in  Huntsville,  Alabama, 
and  the  Boston  Branch  Office  in  Boston,  Massachusetts — to  obtain  the 


4  AMCOM  is  a  major  subordinate  command  of  the  Army  Materiel  Command. 

5  GAO,  Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs,  GAO-10-388SP 
(Washington,  D.C.:  Mar.  30,  2010). 
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customers’  perspectives  on  DCAI  training  and  determine  the  use  of 
supplementary  training.  We  did  not  examine  the  appropriateness  of  the 
certification  training  itself  nor  the  content  of  courses  required  for 
certification.  We  did  not  assess  the  efficacy  of  training  provided  by 
supplementary  training  sources. 

To  assess  the  extent  to  which  acquisition  training  reaches  appropriate 
acquisition  personnel,  we  reviewed  DAU  and  DCAI  policies,  and  we 
received  briefings  from  DAU  and  DCAI  personnel  concerning  the 
determination  of  training  requirements,  resource  allocation,  and 
scheduling  of  classes.  We  reviewed  and  analyzed  the  training  requirements 
for  all  defense  acquisition  career  fields.  We  collected  and  analyzed  defense 
acquisition  workforce  and  training  data  maintained  in  the  AT&L  Data  Mart 
system  used  by  DAU  for  determining  course  demand  and  certification 
status  of  acquisition  workforce  members.  This  provided  an  understanding 
of  the  number  of  class  requests  received,  class  seats  scheduled,  and 
students  who  registered  and  completed  these  courses  in  past  fiscal  years. 
We  also  used  these  data  to  analyze  the  number  and  reasons  for  uncertified 
acquisition  workforce  personnel.  We  assessed  the  reliability  of  these  data 
by  reviewing  data  query  information  for  specific  data  requests  and 
interviewed  knowledgeable  officials  who  collect  and  use  these  data.  We 
intended  to  focus  all  analysis  of  data  for  fiscal  years  2005  through  2009; 
however,  due  to  data  reliability  concerns,  we  limited  portions  of  our 
analysis  to  data  available  for  fiscal  years  2007  through  2009.  We 
determined  that  data  were  sufficiently  reliable  for  the  purposes  of  this 
report.  We  conducted  interviews  with  DAU,  DCAI,  military  department, 
and  defense  agency  representatives  who  have  a  role  in  communicating  or 
analyzing  training  requirements  demand  and  training  resource  allocation 
to  gain  a  fuller  understanding  of  the  processes  and  challenges  faced  when 
providing  training  for  the  defense  acquisition  workforce.  In  addition,  we 
conducted  interviews  with  acquisition  workforce  members  and 
supervisors  to  understand  the  degree  to  which  they  are  able  to  enroll  in 
needed  acquisition  training  and  challenges  they  may  face  in  completing 
this  training.  We  interviewed  DAU  officials  and  obtained  budget 
documents  to  determine  DOD’s  use  of  the  Defense  Acquisition  Workforce 
Development  Fund  (Section  852  of  the  National  Defense  Authorization  Act 
for  Fiscal  Year  2008)  for  training  and  for  helping  to  meet  training  demand. 

To  determine  the  extent  to  which  training  recommendations  from 
previous  reviews,  including  the  Gansler  Commission,  have  been 
implemented,  we  identified  previous  reviews  with  training 
recommendations,  and  we  interviewed  and  obtained  documentation  from 
agency  officials  on  the  status  of  DOD’s  implementation  of  the 
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recommendations.  Specifically,  for  Gansler  Commission 
recommendations,  we  interviewed  Defense  Procurement  and  Acquisition 
Policy  (DPAP)  officials  to  determine  the  applicability  of  the  training 
recommendations  to  the  acquisition  workforce,  and  we  obtained  the 
Office  of  the  Secretary  of  Defense’s  (OSD)  and  the  Army’s  status  in 
implementing  the  recommendations  and  supporting  documents,  including 
reports  detailing  the  recommendations  and  action  items.  We  analyzed  the 
supporting  documents  to  assess  the  status,  and,  based  on  our  review,  we 
assigned  one  of  the  following  six  status  assessments  to  each  of  the 
recommendations. 

(1)  Fully  Implemented.  The  entire  wording  of  the  action  item  has  been 
fulfilled. 

(2)  Partially  Implemented.  Only  a  portion  of  the  action  has  been 
implemented.  When  the  wording  of  the  action  item  had  multiple  parts,  if 
one  part  or  a  portion  of  a  part  had  been  implemented  (but  not  all  parts), 
we  categorized  the  action  item  as  “partially  implemented.” 

(3)  Not  Implemented-Action  Taken.  No  part  of  the  action  item  has  been 
implemented,  but  steps  have  been  taken  toward  the  completion  of  the 
action  item.  For  example,  if  legislation  had  been  introduced  to  address  the 
action  but  had  not  been  enacted  into  law,  we  categorized  the  action  item 
as  “not  implemented-action  taken.” 

(4)  Not  Implemented-No  Action.  No  part  of  the  action  item  has  been 
completed,  and  no  action  has  been  taken  to  address  the  action  item.  For 
example,  if  the  action  item  called  for  changes  in  legislation  but  no 
legislation  has  even  been  proposed,  we  categorized  the  action  item  as  “not 
implemented-no  action.” 

(5)  Insufficient  Information.  Insufficient  or  conflicting  information 
prevented  us  from  determining  the  status  of  the  action  item. 

(6)  Other.  Implementation  has  occurred  or  action  has  been  taken  that, 
while  not  responsive  to  the  letter  of  the  action  item,  generally  was 
consistent  with  its  purpose.  For  example,  if  the  action  item  states  that  a 
particular  position  should  be  created  to  coordinate  an  effort  but  the 
coordination  is  achieved  without  the  creation  of  the  position,  we 
categorized  the  action  item  as  “other.” 

We  compared  our  assessment  with  OSD’s  and  the  Army’s  assessment,  and, 
in  making  our  final  determination  on  implementation  status,  we  provided 


Page  35 


GAO-11-22  Defense  Acquisition  Workforce 


Appendix  I:  Scope  and  Methodology 


OSD  and  Army  officials  the  results  of  our  initial  determinations.  The 
officials  reviewed  these  results  and  provided  us  with  additional,  clarifying 
information  that  we  considered  and,  when  we  believed  appropriate,  used 
in  making  our  final  determination.  For  the  Panel  on  Contracting  Integrity 
reports,  we  examined  whether  DOD  had  implemented  the  Panel’s 
recommendations  in  2007  and  2008  by  reviewing  the  2007,  2008,  and  2009 
reports.  Specifically,  we  compared  the  recommended  actions  from  the 

2007  report  with  the  reported  action  in  the  2008  report.  The  same 
comparative  analysis  was  conducted  using  the  recommended  actions  from 

2008  and  the  2009  report.  We  differentiated  between  recommendations 
that  specifically  mention  training  from  those  that  did  not,  as  well  as 
recommendations  in  which  training  was  involved  in  the  implementation  of 
the  recommendation.  We  compared  our  assessment  with  the  Panel’s 
assessment.  We  provided  our  analysis  to  DPAP  officials  to  review  and 
provide  additional  information  that  we  considered  in  making  our  final 
determination.  To  determine  whether  DCAA  has  implemented  GAO’s 
recommendations  from  a  prior  report,  we  interviewed  officials  at  DCAA  to 
understand  what  actions  had  been  initiated  in  response  to  our 
recommendations.6 

We  conducted  this  performance  audit  from  December  2009  to  September 
2010  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 


6  GAO,  DCAA  Audits:  Widespread  Problems  with  Audit  Quality  Require  Significant 
Reform,  GAO-09-468  (Washington,  D.C.:  Sept.  23,  2009). 
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Planning/front-end  analysis 

(a)  T raining  goals  are  consistent  with  its  overall  mission,  goals, 
and  culture 

(b)  Has  strategic  and  annual  performance  planning  processes 
that  incorporate  human  capital  professionals 

(c)  Determines  the  skills  and  competencies  of  its  workforce 

(d)  Identifies  the  appropriate  level  of  investment  to  provide  for 
training 

(e)  Has  measures  to  assess  the  contributions  that  training  efforts 
make  toward  individual  mastery  of  learning 

(f)  Incorporates  employees'  developmental  goals  in  its  planning 
processes 

(g)  Integrates  the  need  for  continuous  and  lifelong  learning  into 
its  planning  processes 

(h)  Considers  governmentwide  reforms  and  other  targeted 
initiatives  to  improve  management  and  performance  when 
planning  its  training  programs 

(i)  Has  a  formal  process  to  ensure  that  strategic  and  tactical 
changes  are  promptly  incorporated  into  training 

Implementation 

(a)  Agency  leaders  communicate  the  importance  of  training  and 
developing  employees,  and  their  expectations  for  training 
programs 

(b)  Has  a  training  and  performance  organization  that  is  held 
accountable,  along  with  the  line  executives,  for  the  maximum 
performance  of  the  workforce 

(c)  Agency  managers  are  responsible  for  reinforcing  new 
behaviors,  providing  useful  tools,  and  identifying  and  removing 
barriers  to  help  employees  implement  learned  behaviors  on  the 
job 

(d)  Selects  employees  (or  provides  the  opportunity  for 
employees  to  self  select)  to  participate  in  training  and 
development  efforts 

(e)  The  agency  considers  options  in  paying  for  employee  training 
and  development  and  adjusting  employee  work  schedules  so 
that  employees  can  participate  in  these  developmental  activities 

(f)  Takes  actions  to  foster  an  environment  conducive  to  effective 
training 

(g)  Takes  steps  to  encourage  employees  to  buy  into  the  goals  of 
training  efforts 

(h)  Collects  data  during  implementation  to  ensure  feedback  on 
its  training  programs 


Design/development 

(a)  Ensures  that  training  is  connected  to  improving  individual  and 
agency  performance  in  achieving  specific  results 

(b)  The  design  of  the  training  program  is  integrated  with  other 
strategies  to  improve  performance  and  meet  emerging  demands 

(c)  Uses  the  most  appropriate  mix  of  centralized  and  decentralized 
approaches  for  its  training 

(d)  Uses  criteria  in  determining  whether  to  design  training  programs 
in-house  or  obtain  from  a  contractor  or  other  external  sources 

(e)  Compares  the  merits  of  different  delivery  mechanisms  (such  as 
classroom  or  computer-based  training)  and  determines  what  mix  to 
use  to  ensure  efficient  and  cost-effective  delivery 

(f)  Determines  a  targeted  level  of  improved  performance  in  order  to 
ensure  that  the  cost  of  a  training  program  is  appropriate  to  achieve 
the  anticipated  benefit 

(g)  Incorporates  measures  of  effectiveness  into  courses  it  designs 


Evaluation 

(a)  Systematically  plans  for  and  evaluates  the  effectiveness  of  its 
training  efforts 

(b)  Uses  the  appropriate  analytical  approaches  to  assess  its 
training 

(c)  Uses  performance  data  (including  qualitative  and  quantitative 
measures)  to  assess  the  results  achieved  through  training  efforts 

(d)  Incorporates  evaluation  feedback  into  the  planning,  design,  and 
implementation  of  its  training  efforts 

(e)  Incorporates  different  perspectives  (including  those  of  line 
managers  and  staff,  customers,  and  experts  in  areas  such  as 
financial,  information,  and  human  capital  management)  in 
assessing  the  impact  of  training  on  performance 

(f)  T racks  the  cost  and  delivery  of  its  training  programs 

(g)  Assesses  the  benefits  achieved  through  training  programs 

(h)  Compares  its  training  investment,  methods,  or  outcomes  with 
those  of  other  organizations  to  identify  innovative  approaches  or 
lessons  learned 


Source:  GAO,  Human  Capital:  A  Guide  for  Assessing  Strategic  Training  and  Development  Efforts  in  the  Federal  Government, 
GAO-04-546G  (Washington,  D.C.:  Mar.  2004). 
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DEPARTMENT  OF  DEFENSE 

DEFENSE  ACQUISITION  UNIVERSITY 
OFFICE  OF  THE  PRESIDENT 
9820  BELVOIR  ROAD 
FORT  BELVOIR,  VA  22060-5565 


OCT  2  7  2010 

Mr.  John  K.  Needham 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Needham: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report,  GAO-1 1  -22, 
“Defense  Acquisition  Workforce:  Training  for  Defense  Acquisition  Workforce  Demonstrates 
Many  Attributes  of  Effective  Training  but  Can  Be  Improved,”  dated  September  24, 2010  (GAO 
Code  120870). 

The  Department  does  not  concur  with  the  report’s  first  recommendation  and  concurs  with 
the  report’s  second  recommendation.  Detailed  comments  on  both  recommendations  are  enclosed. 

Sincerely, 

James  S.  McMichael 
Acting  President 

Enclosure: 

As  stated 
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GAO  Draft  Report  Dated  SEPTEMBER  24, 2010 
GAO- 11 -22  (GAO  CODE  120870) 

“DEFENSE  ACQUISITION  WORKFORCE:  TRAINING  FOR  DEFENSE 
ACQUISITION  WORKFORCE  DEMONSTRATES  MANY  ATTRIBUTES  OF 
EFFECTIVE  TRAINING  BUT  CAN  BE  IMPROVED” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 


RECOMMENDATION  1:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
USD(AT&L)  to  establish  milestones  for  the  development  of  metrics  to  measure  how  acquisition 
certification  training  improves  the  proficiency  and  capability  of  the  acquisition  workforce.  (See 
page  29/GAO  Draft  Report.) 

DoD  RESPONSE:  Non  concur.  DoD  agrees  that  there  should  be  metrics  to  measure  the 
capability  of  the  Defense  Acquisition  Workforce.  However,  the  establishment  of  milestones  for 
the  development  of  metrics  is  not  necessary  because  existing  metrics  can  be  used  for  this 
purpose.  Taken  together,  the  five  metrics  shown  below  measure  proficiency  and  capability  of 
the  acquisition  workforce. 

Workforce  capability  is  a  function  of  having  the  right  number  of  people  in  the  right  functional 
areas  with  the  right  education,  training,  and  experience.  The  chosen  metrics  address  these 
elements  of  capability  in  terms  of  size  and  composition  and  in  terms  of  individual  education, 
training,  and  experience. 

Workforce  Size  and  Composition 

In  April  2009,  the  Secretary  of  Defense  announced  a  time-phased  growth  strategy  for  the 
Defense  Acquisition  Workforce — grow  by  20,000  people  by  2015.  Responding  to  significantly 
increased  acquisition  workload  and  a  loss  of  critical  skill  sets,  this  initiative  was  aimed  at 
strengthening  organic  core  capability,  particularly  in  areas  of  systems  engineering,  program 
management,  logistics,  contracting,  pricing,  and  cost  estimating.  DoD  has  used  authority  granted 
in  Section  852  of  the  2008  National  Defense  Authorization  Act  to  hire  people  with  the  right 
skills  sets  to  meet  the  secretary's  growth  targets.  The  DoD  Strategic  Management  Plan  contains 
two  metrics  related  to  workforce  growth. 

DoD  Strategic  Management  Plan  Metrics 

•  Cumulative  increase  in  the  number  of  DoD  civilian  and  military  end  strengths 
performing  acquisition  functions. 

•  Cumulative  number  of  DoD  civilian  and/or  military  authorizations  added  as  a 
result  of  in-sourcing  acquisition  functions. 


Workforce  Quality — Education,  Training,  and  Experience 

In  1990  Congress  passed  the  Defense  Acquisition  Workforce  Improvement  Act,  which  required 
that  DoD  improve  the  quality  of  the  acquisition  workforce  by  establishing  education,  experience. 
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and  training  requirements  for  each  acquisition  position.  DoD  responded  to  this  requirement  by 
developing  a  certification  construct  that  incorporated  these  three  components.  Responsible  for 
setting  certification  standards  and  for  oversight,  DoD-appointed  functional  leaders  and  functional 
integrated  product  teams  for  each  acquisition  functional  area  ensure  their  respective  career  fields 
maintain  quality  and  relevance. 

The  DoD  Strategic  Management  Plan  contains  two  metrics  relating  to  certification. 

DoD  Strategic  Management  Plan  Metrics 

•  Annual  improvement  in  percent  of  acquisition  positions  filled  with  personnel 
meeting  or  exceeding  Level  il  certification  requirements. 

•  Annual  improvement  in  percent  of  acquisition  positions  filled  with  personnel 

meeting  or  exceeding  Level  III  certification  position  requirements. _ 


Education  and  Experience :  Functional  leaders  review  the  certification  education  and 
experience  requirements  on  a  continuous  basis  and  change  requirements  as  necessary  to  enhance 
the  effectiveness  and  capability  of  the  acquisition  workforce.  For  example,  the  experience 
requirement  for  the  Systems  Planning,  Research,  Development,  and  Engineering  -  Program 
Systems  Engineer  career  path  was  expanded  from  four  to  eight  years.  This  places  greater 
emphasis  on  experience  as  a  critical  element  in  improving  workforce  quality  and  capability. 
Another  example  is  the  creation  of  a  new  Cost  Estimating  career  path,  which  now  requires  a 
bachelor  degree  and  seven  years  of  experience  to  achieve  Level  III  certification. 

Training :  DoD  has  an  extensive  workforce  training  requirements  process  in  place  which 
involves  functional  leaders  and  joint  functional  teams.  The  joint  functional  teams  develop/revise 
course  requirements  to  ensure  alignment  of  training  with  emerging  DoD  acquisition  policy  and 
initiatives.  These  teams  also  provide  a  rigorous,  ongoing  quality  assessment  of  Defense 
Acquisition  University  (DAU)  course  offerings.  This  expert-based  workforce  training 
requirements  process  is  strengthened  by  a  data-driven  competency-based  approach.  Once 
training  requirements  are  determined,  the  curricula  developers  at  DAU  create  new  courses  or 
update  existing  courses  to  meet  the  newly  established  requirements.  DAU  uses  a  state-of-the-art, 
end-of-course  survey  program.  Metrics  that  Matter  (MTM),  to  assess  the  quality  of  its  courses 
and  other  learning  assets.  MTM  is  a  web-based  learning  evaluation  system  with  an  extensive 
database  of  performance  benchmarks  gathered  from  the  private  and  public  sectors.  In  addition  to 
an  extensive  end-of-course  survey  system,  DAU  also  follows  up  with  students  and  their 
supervisors  to  assess  the  impact  of  training  on  job  performance.  MTM  collects  survey  data  from 
all  students  at  the  end  of  all  courses.  Surveys  are  sent  to  students  of  selected  courses  60  days 
after  course  completion  and  also  to  their  supervisors  120  days  after  course  completion  for  their 
assessment  of  the  impact  of  DAU’s  training. 

DAD  Performance  Dashboard  Training  Quality  Metrics 

•  Meet  or  exceed  the  government/industry  benchmark  averages  for  classroom 
and  online  course  delivery  and  for  student  and  manager  training  effectiveness 
follow-up  surveys. 


Page  40 


GAO-11-22  Defense  Acquisition  Workforce 


Appendix  III:  Comments  from  the  Department 
of  Defense 


While  certification  training  is  critical,  it  must  be  reinforced  with  career-long  learning  assets  to 
support  continued  proficiency  and  capability  of  the  workforce.  The  award  winning  AT&L 
Performance  Learning  Model  equips  the  workforce  for  success  through  a  core  training 
foundation  combined  with  online  continuous  learning,  mission  assistance  to  acquisition 
organization  and  teams,  and  online  knowledge  sharing  resources  and  communities.  The 
Performance  Learning  Model  provides  the  student  with  “reach  back”  to  DAU  and  gives  them 
access  to  professors,  acquisition  resources,  and  other  acquisition  professionals. 

The  five  metrics  listed  above  address  the  proficiency  and  capability  issues  raised  in  the  report. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
USD(AT&L)  to  establish  a  timeframe  for  completion  and  ensure  resources  are  available  for 
implementing  an  enterprise-wide,  integrated  student  information  system.  (See  page  30/GAO 
Draft  Report.) 

DoD  RESPONSE:  Concur.  DoD  has  already  established  a  timeline  for  implementation  of  an 
enterprise-wide  student  information  system  and  funding  is  in  place  for  this  effort. 
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Appendix  IV:  GAO  Contact  and  Staff 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  Web  site, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 

MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Ralph  Dawn,  Managing  Director,  dawnr@gao.gov,  (202)  512-4400 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  DC  20548 

Public  Affairs 


Chuck  Young,  Managing  Director,  youngcl@gao.gov,  (202)  512-4800 
U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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